System and processor implemented method for improved image quality and generating an image of a target illuminated by quantum particles

ABSTRACT

According to some embodiments, system and methods for image improvement comprise: receiving a plurality of frames of a given region of interest, the frames comprised of a plurality of pixels; determining, based on a quantum property of the frames, a normalized pixel intensity value for each pixel of each of the plurality of frames; and generating an improved image of the given region of interest based on the plurality of frames and the corresponding normalized pixel intensity values for the frames, the order of the image being two. Also embodiments for generating an image of a target illuminated by quantum entangled particles, such as, photons, are disclosed.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part (CIP) application of andclaims priority to U.S. patent application Ser. Nos. 14/086,463 and14/022,148, filed Nov. 21, 2013 and Sep. 9, 2013, respectively, bothherein incorporated by reference in their entirety. Those applications,in turn, are a continuation-in-part (CIP) application of and claimpriority to U.S. patent application Ser. No. 13/838,249, filed Mar. 15,2013, now U.S. Pat. No. 8,594,455 and U.S. patent application Ser. No.13/247,470 filed Sep. 28, 2011, now U.S. Pat. No. 8,532,427, hereinincorporated by reference in their entirety. This application alsoclaims priority to U.S. Provisional Application No. 61/834,497, titled“System and Method for Image Enhancement and Improvement,” filed on Jun.13, 2013, herein incorporated by reference.

GOVERNMENT INTEREST

The invention described herein may be manufactured, used, and/orlicensed by or for the United States Government without the payment ofroyalties.

BACKGROUND OF THE INVENTION

Image processing is a form of signal processing for which the input isan image, such as a photograph or video frame, and the output is eitherimage or a set of characteristics or parameters related to the image.Forms of image processing include, for example, but are not limited to,face detection, feature detection, medical image processing, computervision (extraction of information from an image by a computer),microscope image processing, etc.

Image resolution relates at least in part to the detail that an imagepossesses. For satellite images, generally speaking, an image isconsidered to be more detailed as the area represented by each pixel isdecreased. As used herein, the term images include digital images,electronic images, film images, and/or other types of images. Camerastaking pictures from great distances, such as aerial photos, may notobtain detailed information about the subject matter. Consequently,subtle or detail information may not present in the images.

An image may be captured by, for example, a monochrome camera, a singlecharge-coupled device (CCD) or complementary metal-oxide semiconductor(CMOS) sensor and the image is formed via the light intensity projectedonto the sensor therein.

In U.S. Pat. No. 7,536,012, to Meyers et al., hereby incorporated byreference, entitled “Entangled Quantum Communications and QuantumImaging,” there is disclosed, inter alia, a quantum imaging system (seeCol. 8, line 50, et seq.) in which the sender sends an image of an imagemask using entangled photons and coincidence measurements to a receiver.The system differs from the conventional quantum imaging set-up in thatpolarization beam splitters are placed in the path of the photons toprovide two channels for each of the sender and the receiver, as shownin FIG. 4 of the '012 patent. On the sender's side, a photon beam issplit by a beam splitter into first and second sub-beams. The firstsub-beam is passed through a mask 164 which creates the image which isdirected through a beam splitter 166 to bucket detectors 168, 170, whichare operatively connected to a coincidence circuit. The second sub-beamis transmitted to the receiver without ever passing through the mask164. In the embodiment of FIG. 4 of the '012 patent, the receiverreceives the second sub-beam and an image of the mask is constructedbased upon photon coincident measurements composited from two photondetectors 168 and 170, also referred to a bucket detectors. The image ofa mask is transmitted via coincidences and the photons transmitting theimage have never encountered the image mask. Because of the somewhatpuzzling nature or circumstances of the transmission, the process hasbeen dubbed by some as “Ghost Imaging,” while others have explained theeffects as resulting from the quantum properties of light.

SUMMARY OF THE INVENTION

According to embodiments, a processor implemented method for imageimprovement comprising: receiving a plurality of frames of a givenregion of interest, the frames comprised of a plurality of pixels;determining, based on a quantum property of the frames, a normalizedpixel intensity value for each pixel of each of the plurality of frames;and generating an improved image of the given region of interest basedon the plurality of frames and the corresponding normalized pixelintensity values for the frames, the order of the image being two.

The frames of a region of interest are readily influenced by the effectsof turbulence, obscuration, low signal to noise ratio, bad or changingweather and/or low-lighting conditions. This leads to a poor image. Yet,by employing the novel image improvement method of the presentinvention, the image can be significantly improved ameliorating thesenegative effects. Unlike classical image improvement techniquesconventionally-employed, the novel processing relies upon quantumproperties.

The received frames may be generated by the detector—directly orindirectly. For instance, the frames could be output straight from thedetector for processing, or the frame may be generated by the detector,stored in a memory, and retrieved from the memory for processing at somelater time. Not all frames need to be used for processing for allapplications. Indeed, in some implementations, fewer than the totalnumber of frames can be used to determine an improved image of theregion of interest.

The normalized pixel intensity value may be determined based on aquantum property of the same frame data which comes from the detector.As such, only one detector or measuring device may be needed in someembodiments. In general terms, the normalized pixel intensity values maybe thought as of averaged intensity values. For instance, determiningthe normalized pixel intensity value of a frame may comprise:determining pixel values within a frame; summing pixel intensity valuesfor determined pixel values within a frame; and dividing each summedpixel value by the number of determined pixel values to form anormalized pixel intensity value for each pixel in the frame.

Generating the improved image of the region of interest can thencomprise: calculating (i) the average of the product of determined pixelvalues and the corresponding normalized pixel intensity values for theplurality of frames, and (ii) the product of the average of thedetermined pixels values for each frame and the average of normalizedpixel intensity values for the plurality of frames. And, in someembodiments, generating the improved image of the region of interestcomprises may include further comprising: taking the difference of (i)and (ii).

Calculating (i) the average of the product of determined pixel valuesand the corresponding normalized pixel intensity values for theplurality of frames, in some instances, may comprise: multiplying pixelvalues for determined pixels within each frame by the correspondingnormalized pixel intensity values for that frame to produce a productfor each frame; summing the products of all the frames; and determiningthe average of first product arrays by dividing the sum of product bythe number of frames. And, calculating (ii) the product of the averageof the determined pixels values for each frame and the average ofnormalized pixel intensity values for the plurality of frames, in someinstances, may comprise: determining the average value of each pixel foreach frame for the plurality of frames; determining the averagenormalized pixel intensity value for each pixel for the plurality offrames; and multiplying the average pixel values and the average of thenormalized pixel intensity value for each pixel.

Determining pixels values within a frame for processing may be achievedby various methodologies, including one or more of: selecting all pixelswithin each frame; selecting pixel based upon at least one predeterminedcriterion; selecting pixels which are shifted a pre-determined distanceaway from select pixels; and/or determining an average value of adjacentpixels for select pixels. Practicing the image improvement method mayfurther includes selecting at least one measureable property fordetermining a normalized pixel intensity value for each pixel of each ofthe plurality of frames; and using at least one different measurableproperty of the plurality of frames for generating the improved image.

A measureable property may include: wavelength or wavelength band,color, polarity, polarization, orbital angular momentum, spin, a quantumparticle; or any combination thereof.

Depending on the desired application, the frames can comprise regions ofinterest that are radiation emitting. Or the frames of a region ofinterest comprise sparse image data, and the improved image is generatedusing the sparse image data.

Select processing in generating the improvement image may be furtheremployed, in some embodiments, based on differences among the frames.For instance, the image improvement method can further include:determining a frame intensity deviation value for each frame bysubtracting the average frame intensity for the plurality of firstframes from the frame intensity for each frame; and classifying theframe intensity deviation values for each frame based on whether theframe intensity deviation values is positive or negative. Then selectingprocessing for generating an improved image may be carried out based onthe aforementioned classification. In addition, select processing may becarried out based on conditional product values for pixels. For example,the method can further comprise: calculating one or more conditionalproduct values of the classified frame intensity deviation values foreach frame; and selecting one or more of the conditional product valuesto generate the improved image. More, in some embodiments, at least twocalculated conditional product values are treated differently based upontheir classification. Or, all calculated conditional product valuesmight be used to generate the improved image without any change thereto.

Additional refinements of the image improvement method may further beemployed in many embodiments. These may comprise interpolating the pixelvalues for each frame to a finer resolution. Also, filtering of theframe data, the normalized pixel intensity value, and/or any data usedin one or more calculations thereof, can be utilized to additionalimprove the generation of the image.

Iterating or repeating certain processing may also be employed inembodiments. For instance, providing an iterated improved image of theregion of interest may comprises: specifying one or more pixel locationsto be normalized to form the normalized pixel intensity value for eachpixel; selecting new pixel locations based on a pre-determined pixelselection criteria from the values of the improved image of the regionof interest; reviewing the new determined pixels to determine if the newdetermined pixel locations are substantially the same as the pixellocations previously determined pixel locations; and repeating theaforementioned steps until a specified iteration criteria is met.

According to embodiments, a system for image improvement comprises: atleast one processor; and at least one input for receiving or inputtingframes of data; and at least one memory operatively associated with theat least one processor adapted to store frames of data taken of a regionof interest, each frame of data comprises an array of pixels, with eachpixel having a pixel value. The at least one processor is configured toprocess a plurality of frames of a given region of interest according toone or more or the aforementioned image improvement method embodiments.The input may be operably connectable to an input device, such as, ascanner, a DVD player, CMOS camera, SPAD array, video camera, smartphone, plenoptic camera, cell phone, lidar, ladar, television, CCD oranalog and/or digital camera.

According to other embodiments, a processor implemented method for imageimprovement comprises: receiving a plurality of frames of a given regionof interest, the frames comprised of a plurality of pixels, each pixelincluding a value of at least one measureable property of quantumparticles; specifying the order of the improved image to be generated,the order being greater than or equal to two; selecting at least onemeasureable quantum property for pixel values of the framescorresponding to the specified order; determining, based on the at leastone measureable quantum property, normalized pixel intensity values foreach pixel of each of the plurality of frames up to the specified order,to generate the improved image; and generating an improved image of thegiven region of interest based on the plurality of frames and thecorresponding normalized pixel intensity values for the frames. The atleast measureable quantum property can comprise, for example: wavelengthor wavelength band, color, polarity, polarization, orbital angularmomentum, spin, quantum phase, a quantum particle; or any combinationthereof.

Additionally, according to embodiments, a system for generating an imageof a target illuminated by quantum entangled particles. The systemincludes: an illumination system comprising: at least one source ofquantum entangled particle pairs; and a beamsplitter receiving thequantum entangled particle pairs, such that one particle from each pairof particle generated by each source interfere on the beamsplittercausing the interfered particles to be directed towards a target and theremaining particle pairs are not directed towards the target, whereinthe illumination system is configured so that the interfered particlesinteract with the target; a measuring system comprising a first detectorand a second detector that are configured to perform at least onespatially resolved measurement of particles, where the first detectormeasures one of the remaining particle pairs and the second detectormeasures the other of the remaining particle pairs; and a processorconfigured to generate an image of the target based upon the correlatedmeasured values and spatial information from the first detector and thesecond detector.

In some implementations, the system can further include: electronicsconfigured to determine coincidences based on measurements of the firstand second detectors which occur within a predetermined time interval.The processor can further be configured to generate at least a secondorder image using the coincidences. Also, the processor may beconfigured to apply an image improvement method for generating at leasta second order image using at least one measureable quantum property.

In other implementations, the system may further include an opticaldelay element configured to introduce a time delay for particlesreaching the measuring system. This optical delay element may, in someembodiment, be further configured to be operated so as to generate anabsorption image of the target, a reflection image of the target, orboth.

Additionally, the system may optionally include is some embodiment anoptical delay line configured to ensure particle interference at thebeamsplitter, and/or a phase modulator configured to modify the phaserelationship between the particle pairs generated by the two sources ofquantum entangled particle pairs, respectively.

In one particular embodiment, the illumination system comprises: asingle source of entangled particle pairs; and a pair of beamsplittersreceiving the entangled particles pairs, such that one particle fromeach pair of particles generated by each source interfere on thebeamsplitter causing the interfered particles to be directed towards atarget and the remaining particle pairs to be retained; wherein theillumination system is configured so that the interfered particlesinteract with the target causing absorption at the target entangling theretained particle pair.

In some embodiments, the system may be configured so that the interferedparticles interact with the target causing absorbtion at the targetentangling the retained particle pairs. And, it yet other embodiments,the system may be configured so that the interfered particles interactwith the target causing reflection at the target and further comprisesoptics or focusing components and measurement electronics wherein themeasurement of the reflected entangled particles entangled the retainedparticle pairs.

These and other aspects of the embodiments herein will be betterappreciated and understood when considered in conjunction with thefollowing description and the accompanying drawings. It should beunderstood, however, that the following descriptions, while indicatingpreferred embodiments and numerous specific details thereof, are givenby way of illustration and not of limitation. Many changes andmodifications may be made within the scope of the embodiments hereinwithout departing from the spirit thereof, and the embodiments hereininclude all such modifications.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention can best be understood when reading the followingspecification with reference to the accompanying drawings, which areincorporated in and form a part of the specification, illustratealternate embodiments of the present invention, and together with thedescription, serve to explain the principles of the invention. In thedrawings:

FIG. 1 is a partial schematic block diagram illustration of the stepsfor performing a preferred method of the present invention.

FIG. 2 is a schematic block diagram illustration of the steps forperforming a preferred method of the present invention. Taken together,FIGS. 1 and 2 outline the steps of a preferred methodology for thepresent invention.

FIG. 3 is a partial schematic block diagram illustration of the stepsfor performing an alternate preferred method of the present invention inwhich steps for dividing the frames into two sets are illustrated.

FIG. 4 is a partial schematic block diagram illustration of the stepsfor performing an alternate preferred method of the present invention inwhich steps performed on the first set of frames are illustrated.

FIG. 5 is a partial schematic block diagram illustration of the stepsfor performing an alternate preferred method of the present invention inwhich steps performed on the second set of frames are illustrated.

FIG. 6 is a partial schematic block diagram illustration of the stepsfor performing an alternate preferred method of the present invention inwhich the refined image data for the second set is subtracted from therefined image data for the first set.

FIG. 7 is an illustration of a G⁽²⁾ Virtual Ghost Image with turbulenceusing 10 k frames.

FIG. 8 is an illustration of a G⁽²⁾ Ghost image computed using the 2path configuration.

FIG. 9 is an illustration of the “Mean Bucket/Target Image” using 10 kframes.

FIG. 10 is an illustration of a “Self Bucket G⁽²⁾ GPSR with turbulence”using 10 k Frames; data normalized globally 0-1; τ=1×10⁸; Tol=1×10⁻⁶;Non Zero=67 (number of pixels not zero).

FIG. 11 is an illustration of a “Self Bucket G(2) GPSR” with turbulence10 k Frames; Data normalized globally 0-1; τ=5×10⁷; Tol=1×10⁻⁶; NonZero=131.

FIG. 12 is an illustration of a “Self Bucket G(2) GPSR” with turbulence10 k Frames; Data normalized globally 0-1; τ=2.5×10⁷; Tol=1×10⁻⁶;NonZero=183.

FIG. 13 is an illustration of a “Self Bucket G(2) GPSR” with turbulence10 k Frames; Data normalized globally 0-1; τ=1×10⁷; Tol=1×10⁻⁶; NonZero=304.

FIG. 14 is an illustration of a “Self Bucket G(2) GPSR” with turbulence10 k Frames; Data normalized globally 0-1; τ=1×10⁶; Tol=1×10⁻⁶; NonZero=1310.

FIG. 15 is an illustration of a sample instantaneous data image.

FIG. 16 is an illustration of an average of 335 frames.

FIG. 17A is an illustration of an image formed utilizing the Ghostimaging concept using 2 frames taken at a distance of 100 m throughturbulence.

FIG. 17B is an illustration of an image formed using 335 frames; “SelfBucket G⁽²⁾, 100 m distance through turbulence.

FIG. 18 depicts a high level block diagram of a general purpose computerconfigured to implement embodiments of the present invention.

FIG. 19A is a schematic block diagram of an alternate preferredembodiment.

FIG. 19B is a schematic block diagram of an alternate preferredembodiment similar to FIG. 19A but further including, inter alia, achannel 129 for transferring the measured bucket values to theprocessor.

FIG. 20A is a schematic block diagram illustration of an alternatepreferred embodiment of the present invention wherein groups andsubgroups are used to determine product arrays to form an improvedimage.

FIG. 20B is a continuation of the schematic block diagram illustrationof FIG. 20A.

FIG. 20C is a continuation of the schematic block diagram illustrationof FIG. 20A.

FIGS. 21-31 are schematic block diagram illustrations of the steps foran alternate preferred embodiment to compute the fluctuation, ordeviation from the mean value of the series of “bucket” measurementsaccording to the alternate preferred embodiment.

FIG. 21 is schematic block diagram illustration of the steps to computethe fluctuation, or deviation from the mean value of the series of theper frame pixel measurements.

FIG. 22 is a partial schematic block diagram illustration which showshow to generate a third set of data which is referred to here at SET 3.The illustrated steps are performed on the above mean, above mean setsof frames.

FIG. 23 is a partial schematic block diagram illustration that is acontinuation of FIG. 22.

FIG. 24 is a partial schematic block diagram illustration showing how togenerate a fourth set of data which is referred to here at SET 4. Theillustrated steps are performed on the below mean, below mean sets offrames.

FIG. 25 is a partial schematic block diagram illustration that is acontinuation of FIG. 24.

FIG. 26 is a partial schematic block diagram illustration of the stepshow to generate a fifth set of data which is referred to here at SET 5.The illustrated steps are performed on the above mean, below mean setsof frames.

FIG. 27 is continuation of the schematic block diagram illustration ofFIG. 26.

FIG. 28 is a partial schematic block diagram illustration showing how togenerate a sixth set of data which is referred to here at SET 6. Thesteps are performed on the below mean, above mean sets of frames.

FIG. 30 is a partial schematic block diagram illustration of the stepsfor performing an alternate preferred method of the present invention inwhich the improved final image is determined by adding the above-mean,above mean image to the below-mean, below-mean images, subtracting theabove-mean, below-mean image, and subtracting the below-mean, above meanimage.

FIG. 31 is a partial schematic block diagram illustration of the stepsfor performing an alternate preferred method of the present invention inwhich the improved final image is determined by adding the above-mean,above mean image, the below-mean, below-mean image, the above-mean,below-mean image, and the below-mean, above mean image.

FIG. 32 is an illustration showing results using a standard G⁽²⁾calculation to provide a baseline to demonstrate the advantages of themethods presented in this disclosure.

FIG. 33 is an illustration showing the improved imaging results whenapplying the methods and processes outlined in FIG. 30. The trees in thedistance and clouds are much more distinguishable in this figure whencompared to the results shown in FIG. 32.

FIG. 34 is an illustration showing the improved imaging results usingthe methods and processes outlined in FIG. 31. As in FIG. 33, the treesand clouds are much more distinguishable than what is seen in FIG. 32.

FIG. 35 is an illustration showing the image contract improvement,wherein both the Log positive and negative components of the base G⁽²⁾image show increased contrast and sharpening of edges especially whencompared to the average image in the lower left.

FIG. 36 is an illustration showing improved image results generated,wherein features such as the lamp post show much more contrast and edgeclarity.

FIG. 37 is an illustration showing results of an enhanced imagegenerated using preferred embodiment of FIGS. 20A-20C with data acquiredin low-light and turbulence conditions viewed from 2.33 kilometers.

FIG. 38 is an illustration showing an average image of target area withdata acquired in low-light and turbulence conditions viewed from 2.33kilometers.

FIG. 39 is a view of the target shown in FIGS. 37 and 38 from a shortdistance.

FIG. 40 is schematic block diagram illustration of the steps forperforming an alternate preferred method of the present invention.

FIG. 41 is a schematic block diagram illustration of the steps forperforming an alternate method of the present invention. Taken together,FIGS. 40 and 41 outline the steps of an alternate preferred methodologyfor the present invention.

FIG. 42 is a schematic diagram of an alternate preferred embodimentinvolving coordinate shifting of pixel values.

FIG. 43 a partial schematic block diagram illustration of the steps forperforming an alternate preferred method involving coordinate shiftingof pixel values.

FIG. 44 is a schematic block diagram illustration of the steps forperforming an alternate preferred method involving coordinate shiftingof pixel values. Taken together, FIGS. 43 and 44 outline the steps of analternate preferred methodology involving coordinate shifting of pixelvalues.

FIG. 45 is a partial schematic block diagram illustration of analternate preferred embodiment method involving a generalized means toselect pixels to use as “bucket” values for each pixel in the pluralityof frames of data.

FIG. 46 is a schematic block diagram illustration of the steps forperforming an alternate preferred embodiment involving selecting a setof pixels to generate a normalized sum to use as a “bucket” value foreach measured pixel value in the plurality of frames. Taken together,FIGS. 45 and 46 outline the steps of an alternate preferred embodimentinvolving selecting a set of pixels to generate a normalized sum to useas a “bucket” value for each measured pixel value in the plurality offrames.

FIG. 47 is a high level block diagram configured to implement aparticular embodiment of the present invention.

FIG. 48 is a high level schematic block diagram of an alternatepreferred embodiment similar of the present invention.

FIG. 49 is a schematic block diagram of an alternate preferredembodiment using illumination with entangled photons.

FIG. 50 is a schematic block diagram of an alternate preferredembodiment with two telescopes and entangled photon illumination.

FIG. 51 is a schematic block diagram of an alternate preferredembodiment with two telescopes.

FIG. 52 is a schematic flow-chart diagram of an embodiment to perform aninterative process for image improvement.

FIG. 53 is schematic block diagram of the inventive process wheresub-ensembles are used to generate improved images of the region ofinterest.

FIG. 54 is a block diagram of an embodiment of a system for imageimprovement.

FIG. 55 is an exemplary 3rd order enhanced imaging method embodiment.

FIG. 56 is another exemplary 3rd order enhanced imaging methodembodiment.

FIG. 57 is an exemplary 4^(th) order enhanced imaging method embodiment.

FIGS. 58-67 are various images for comparison sake in accordance withembodiments of the present invention.

FIG. 68 is a sample all positive G⁽³⁾ (R+++ conditional product term)image of a distant target.

FIG. 69 is a sample G⁽²⁾ image generated by shifting the pixels of theimage shown in FIG. 70 by ten pixels and using as a bucket the pixelvalues located at (130,15).

FIG. 70 is an image of the target area that was shifted by ten pixels.

FIG. 71 is a sample all positive 2^(nd) order image generated using thesame input as FIG. 69.

FIG. 72 is a block diagram for an alternate embodiment of a system forimage and ranging improvement.

FIG. 73 is a comparison of filtered “bucket” values vs. time (dottedline) and unfiltered “bucket” values vs. time (solid line).

FIG. 74 is an example of results generated using seven frames with afiltering technique on the input measurements.

FIG. 75 is an example of results generated using the same seven framesof unfiltered input measurements.

FIG. 76 is an example of the average image generated using a filteringtechnique on the input measurements.

FIG. 77 is an example of the average image generated using unfilteredinput measurements.

FIG. 78 is an exemplary embodiment of a filtering process used onmeasured input values.

FIG. 79 is a block diagram for an embodiment of a system for image andranging improvement which is configured to transmit entangled photonpairs to the region of interest.

FIG. 80 is a schematic block diagram of an embodiment for the generationof related entangled photon pairs.

FIG. 81 is a schematic block diagram of an alternate embodiment for thegeneration of related entangled photon pairs.

FIG. 82 is a schematic block diagram for an alternate embodiment of asystem for image and ranging improvement.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The embodiments herein and the various features and advantageous detailsthereof are explained more fully with reference to the non-limitingembodiments that are illustrated in the accompanying drawings anddetailed in the following description. Descriptions of well-knowncomponents and processing techniques are omitted so as to notunnecessarily obscure the embodiments herein. The examples used hereinare intended merely to facilitate an understanding of ways in which theembodiments herein may be practiced and to further enable those of skillin the art to practice the embodiments herein. Accordingly, the examplesshould not be construed as limiting the scope of the embodiments herein.

Rather, these embodiments are provided so that this disclosure will bethorough and complete, and will fully convey the scope of the inventionto those skilled in the art. Like numbers refer to like elementsthroughout. As used herein the term “and/or” includes any and allcombinations of one or more of the associated listed items.

It will be understood that, although the terms first, second, etc. maybe used herein to describe various groups, subgroups, elements,components, regions, layers and/or sections, these groups, subgroups,elements, components, regions, layers and/or sections should not belimited by these terms. For example, when referring first and secondgroups or subgroups, these terms are only used to distinguish one group,subgroup, element, component, region, layer or section from anothergroup, subgroup, region, layer or section. Thus, a first group,subgroup, element, component, region, layer or section discussed belowcould be termed a second element, component.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to limit the full scope of theinvention. As used herein, the singular forms “a”, “an” and “the” areintended to include the plural forms as well, unless the context clearlyindicates otherwise. It will be further understood that the terms“comprises” and/or “comprising,” when used in this specification,specify the presence of stated features, integers, steps, operations,elements, and/or components, but do not preclude the presence oraddition of one or more other features, integers, steps, operations,elements, components, and/or groups thereof.

Unless otherwise defined, all terms (including technical and scientificterms) used herein have the same meaning as commonly understood by oneof ordinary skill in the art to which this invention belongs. It will befurther understood that terms, such as those defined in commonly useddictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of the relevant art andwill not be interpreted in an idealized or overly formal sense unlessexpressly so defined herein.

The present invention provides a method and system for the enhancementof images using the quantum properties of light or matter. An embodimentof the present invention increases the image quality of an object orscene as seen by a detector. When a low quality detector is aimed at anobject, a high quality image is generated using the quantum propertiesof light. A low quality detector picks up quantum information on theobject shape and its temporal relations to reference fields. Thereference fields may be recorded by the same imager (CCD, camera, etc.)that acts as a bucket detector (that is, it does not necessarily containspatial information). Current imaging methods are limited to the qualityof the detector looking at the object being imaged. A preferredembodiment generates an improved quality image of the object without theobject being imaged in high resolution directly. The preferred methodmay be used in connection with photographs taken during turbulentconditions.

An alternate embodiment comprises computing the average overallintensity of a plurality of frames and arranging the frames into twosets. A first set contains the frames having frame intensities greaterthan the average overall intensity for all frames; the average overallintensity being the summation of the intensities for frames divided bythe number of frames. The second set containing frames having an overallintensity less than the average overall intensity. Each of the first andsecond sets is processed by repeating steps (a) through (i). The resultobtained using the second set of frames is then subtracted from theresult obtained using the first set of frames to create the image.

Yet another embodiment includes a method for image improvement thatcomprises: providing a series of frames of a given region of interest;determining the value of each pixel within each frame to form a firstarray of pixel values; determining the integral of pixel values of eachframe to form a second array of overall frame integrals; partitioningthe first array into two sets based on a predetermined criteria;partitioning the second array into two sets based on a predeterminedcriteria; using the first and second sets of the first and second arraysto compute a conditional product term for each of the four combinationsof the first and second sets of the first and second arrays; andcombining the conditional product terms to provide an improved image ofthe region of interest.

And yet a further embodiment includes a method for image improvementcomprising providing a series of frames of a given region of interest;determining the value of each pixel at each location within each frameto form a first array of pixel values for each frame; determining theoverall intensity of each frame; determining the product of the overallintensity and the array of pixel values for each frame; determining thesum of the products by adding together the products of the overall frameintensity and first array of pixel values for each frame; determiningthe average of the sum of products by dividing the sum of products bythe number of frames in the series of frames; determining the averagevalue of each pixel at each pixel location for the series of frames toform a second array of average pixel values; determining the averageoverall frame intensity for the series of frames; determining a secondproduct of the second array of average pixel values and the averageoverall frame intensity; subtracting the second product from the firstproduct to provide an improved image of the region of interest;partitioning the improved image into at least two partitions based on apredetermined criteria; mathematically operating upon the partitionedimproved image to increase image contrast or image clarity.

The current invention utilizes the ability to increase the image qualityof an object as seen by a detector using methods relating to the Quantumnature of light. For example, when a low quality detector is aimed at anobject, then a high quality image may be generated based on the quantumproperties of light. The high quality image is generated even in thepresence of turbulence which might otherwise be disruptive to imageclarity. Scattering of quantum particles such as photons off the objectcarries information of the object shape even when the quantum particlessuch as photons do not go directly into the camera or detector. Anadditional low quality bucket detector (such as, for example, a detectorlacking spatial information) records quantum information on the objectshape and its temporal relations to collocated reference fields. Thereference fields may be recorded by the same type of imager (CCD,Camera, etc.) that looks at the object and which act like bucketdetectors in U.S. Pat. No. 7,536,012, referenced in the foregoing.

Current imaging methods are limited to the quality of the detectorlooking at the object being imaged. Embodiments of this invention enablean image quality improvement by using techniques developed in the courseof Ghost Imaging experiments, and includes, but is not limited to,methods to generate a high quality image of the object without theobject being imaged in high resolution directly; i.e., low resolutionimages may be enhanced, thereby enabling high quality imaging when onlylow quality images of the object are imaged directly.

Second Order Imaging

Imaging of a scene or subject is typically accomplished by mapping anilluminated scene or subject onto an image sensor where there is a lightmeasurement component such as film, CCD, or other sensing device. Lightconsists of a plurality of photons that may be measured. Theilluminating light may be from one or more light sources either naturalor artificial, or both. Common sources of light include for example thesun, coherent, incoherent, or partially coherent light, entangledphotons, infrared radiation emitted by atoms and molecules, acceleratingcharges, lasers, light bulbs, light emitting diodes (LEDs), chaoticlaser light, pseudo-thermal light generated by passing laser lightthrough a rotating ground glass or other scattering material, stars,moons, clouds, planets, space objects, fluorescent lamps, electricaldischarges, plasmas, bio-luminescence, and stimulated emission. Althoughit is not absolutely necessary a lens is often used to perform thismapping. Imaging is often susceptible to adverse affects such asobscuration, turbulence, low signal to noise ratio such as whenoperating in low-light conditions, jitter, and noise. Often, this typeof imaging is referred to as “First Order” imaging due to the time,ensemble, or mixed time-ensemble averaging of the sensors involved. Forinstance, a first-order light intensity image I(x, y, t) can be producedby light interacting with a sensor for some time Δt, i.e. shutter orintegration time. A single instance of this may be referred to as a“frame”. Multiple frames of images, I(x, y, ti), may be averaged oversome or all of the frames in a sequence of frames to generate anaveraged first-order image of the subject <I(x, y, > where < > indicatesan ensemble average. A second order image involves averages of productsof two first-order intensity or normalized intensity measurements. Anenhanced image results from the subtraction of products of averages offirst order intensities from the average of the product of theintensities. An intensity or normalized intensity can be decomposed intoa mean (<I₁>) plus a deviation from the mean or average (ΔI₁).

The terms I₁ and I₂ are intensities or normalized intensities measuredby sensors 1 and 2, I₁=<I₁>+ΔI₁ and I₂=<I₂>+ΔI₂ with I₁ and I₂ beingfunctions of space and time, i.e., x, y, t. <I₁> is the ensemble averageof intensity or normalized measurements of sensor 1 and ΔI₁ is thedeviation from the mean for the intensity or normalized intensitymeasurements of sensor 1. <I₂> is the ensemble average of intensity ornormalized measurements of sensor 2 and ΔI₂ is the deviation from themean for the intensity or normalized intensity measurements of sensor 2.The deviation is often called a fluctuation.

Mathematically the second-order enhanced image can be described by<I₁I₂>=<<I₁><I₂>+ΔI₁<I₂>+ΔI₂<I₁>+ΔI₁ΔI₂>. Simplifying this expressionyields <I₁I₂>=<I₁><I₂>+<ΔI₁ ΔI₂>.

Similarly

$\begin{matrix}\begin{matrix}{{\langle{\Delta \; I_{1}\Delta \; I_{2}}\rangle} = {\langle{\left( {I_{1} - {\langle I_{1}\rangle}} \right)\left( {I_{2} - {\langle I_{2}\rangle}}\rangle \right.}}} \\{= {\langle{{I_{1}I_{2}} - {I_{2}{\langle I_{2}\rangle}} - {I_{2}{\langle I_{1}\rangle}} + {{\langle I_{1}\rangle}{\langle I_{2}\rangle}}}\rangle}} \\{= {{\langle{I_{1}I_{2}}\rangle} - {2{\langle I_{1}\rangle}{\langle I_{2}\rangle}} + {{\langle I_{1}\rangle}{{\langle I_{2}\rangle}.}}}}\end{matrix} & \; \\{{{\langle{\Delta \; I_{1}\Delta \; I_{2}}\rangle} =}{{\langle{I_{1}I_{2}}\rangle} - {{\langle I_{1}\rangle}{{\langle I_{2}\rangle}.}}}} & \;\end{matrix}$

A higher order enhanced image can be described with similar expressions.For example a 3^(rd) order image can be described by

<ΔI ₁ ΔI ₂ ΔI ₃>=<(I ₁ −<I ₁>)(I ₂ −<I ₂>)(I ₃ −<I ₃>)

<ΔI₁ ΔI₂ ΔI₃>=<I₁I₂I₃>−<I₁><I₂><I₃> and a N^(th) order enhanced imagemay be described by

<ΔI ₁ ΔI ₂ . . . ΔI _(N) >=<I ₁ I ₂ . . . I _(N) >−<I ₁ ><I ₂ > . . . <I_(N)>.

I₁ . . . I_(N) may be selectable by user as follows. For instance, animaging system may be configured to provide at least one measurementvalue for at least one spatial location of a region of interest. Thespatial location can be represented by position within an array of“pixels”. A frame of measured values may be composed of a plurality ofpixels, typically in an at least one-dimensional array that together mayform an image. Exemplary frames may be electronic image data such a TIFFor JPEG file. The measured values can be assigned to variables I₁, I₂, .. . , I_(N) in a variety of ways. As an example, when the imaging systemprovides a single color, or gray scale, image of the region of interestper frame then each measured pixel value may be assigned to variable I₁,and variable I₂ may be assigned the value of the sum of the measurementpixel values per frame. For a different embodiment I₂ may be, forinstance, assigned the measured values of a particular pixel, say pixeli=12, j=300, and variable I₃ assigned the measured values of pixel,i=522, j=207 where i and j are indices into the 2D arrays of measuredvalues. Generally, each measured value assigned to one of the I_(N)variables may be selected by the user to meet the requirements for theirspecific application. At least one of the variables I_(N) must beassigned positions that correspond to the pixel positions of themeasured pixel value arrays provided by the imaging system at knownpixel position coordinates.

In an example of a second order improved image, I₁ and I₂ may refer tointensities measured by at least two sensors where one of the sensorsmeasures spatial information of the light (I₁) coming from the scene orsubject (the “Reference” sensor) and the other sensor measures aquantity representative of the intensity (I₂) coming from the scene orsubject, i.e. a “the bucket” sensor. One of the sensors may be a“virtual” sensor wherein, for instance, the representative intensitycoming from the scene or subject is comprised of spatially integratingall or a selected subset of pixels on a CCD or CMOS camera or evenconsist of a single pixel from a CCD or CMOS camera. The enhanced imageis contained in <ΔI₁ ΔI₂> which has a δ-function like correspondencebetween points on the object and points on the image sensor and islargely unaffected by the degrading effects of turbulence, obscuration,low signal to noise ratio such as when operating in low-lightconditions, jitter, and noise. See for example, Meyers et al.,“Turbulence-free ghost imaging,” Appl. Phys. Lett. 98, 111115, 2011,herein incorporate by reference.

A preferred method for practicing the present invention may becorrelated to the mathematical representations as follows. Expressedusing the terms I₁ and I₂, a preferred method for image improvementcomprises inputting a series of frames of an image; determining thevalue of each pixel at each location within each frame to form a pixelvalue array for each frame; summing the pixel values in each frame toobtain the frame intensity for each frame (correlates to determiningI₂); multiplying the pixels in pixel value array by the frame intensityto produce a frame intensity multiplied pixel value array (correlates todetermining the product I₁I₂); summing the frame intensity multipliedpixel value arrays together and dividing by the number of frames toobtain an average of the frame intensity multiplied pixel value arrays(correlates to determining <I₁I₂>); using the pixel value arrays,creating an array of average pixel values (correlates to determining<I₁>); determining the average frame intensity for the series of frames(correlates to determining <I₂>); multiplying the array of average pixelvalues by the average frame intensity for all of the inputted frames(correlates to the product <I₁><I₂>); and subtracting the array ofaverage pixel values multiplied by average frame intensity (<I₁><I₂>);from the average of the frame intensity multiplied pixel value arrays(correlates to <I₁I₂>) to provide an array of modified pixel values toform an improved image <ΔI₁ ΔI₂> second order image (which, from thepreviously expresses mathematical equations correlates to the equation<ΔI₁ ΔI₂>=<I₁I₂>−<I₁><I₂>).

Other preferred methods may include the normalizing of the intensity toproduce an enhanced image. There are several ways to normalizeintensity. One way is to divide the Reference pixel intensity values bya non-zero value “bucket” intensity, J₁=I₁/I₂. This normalization wouldgive J₁=<J₁>+ΔJ₁ and I₂=<I₂>+ΔI₂ with J₁ and I₂ being functions of spaceand time, i.e. x, y, t. Where J₁ and I₂ are normalized intensities andintensities measured by sensors 1 and 2. <J₁> is the ensemble average ofintensity or normalized measurements of sensor 1 and ΔJ₁ is thedeviation from the mean for the normalized intensity measurements ofsensors 1 and 2. <I₂> is the ensemble average of intensity or normalizedmeasurements of sensor 2 and ΔI₂ is the deviation from the mean for theintensity or normalized intensity measurements of sensor 2. Thedeviation is often called a fluctuation.

Mathematically the second-order enhanced image can be described by<J₁I₂>=<<J₁><I₂>+ΔJ₁<I₂>+ΔI₂<J₁>+ΔJ₁ΔI₂>. Simplifying this expressionyields

<J ₁ I ₂ >=<J ₁ ><I ₂ >+<ΔJ ₁ ΔI ₂>,

rearranging terms yields

<ΔJ ₁ ΔI ₂ >=<J ₁ I ₂ >−<J ₁ ><I ₂>

wherein the enhanced image is contained in <ΔJ₁ ΔI₂>. The enhanced imagemay be normalized by the product of the standard deviations of I₁ and I₂to generate an enhanced image that displays the correlation of I₁ andI₂. Other alternative ways to normalize the enhanced image includedividing <ΔI₁ ΔI₂> or <ΔI₁ ΔI₂> by the product <J₁><I₂> or <I₁><I₂>respectively.

One embodiment of the current invention comprises the subject areaillumination being generated by one or more light sources which can beinternal, external or a mixture of external and internal light sources.Examples of external light sources include the sun, coherent,incoherent, or partially coherent light illuminating the subject areagenerated by natural or artificial means indoors or out of doorspropagating through any transmissive or partially transmissive mediasuch as the air, water, or biological tissue including cells. Examplesof internal light sources include the subject emanating light in theinfrared given off by atoms and molecules. Light received may bereflected, scattered, or emanated from the subject into at least onefirst receiver at predetermined time intervals. Light may be received atthe at least one second receiver at corresponding time intervals fromthe light source which may be reflected or partially reflected from thesubject and contains spatial information. The first and second receiversmay be selected from, for example, one or more arrays of pixels from oneor more cameras, imagers, CCDs, etc. In a preferred embodiment, themeasured values are transmitted from the first and second receivers tothe at least one processor. The measured values of the at least onefirst receiver are then correlated with the spatially resolvedmeasurements of the at least one second receiver at the correspondingintervals of time. A first image of the subject is then created basedupon the correlated measured values and spatial information by combiningthe spatial information from at least one second receiver atpredetermined intervals of time with the measured values from at leastone first receiver at the corresponding intervals of time. An enhancedsecond image of the subject is generated by removing the blurred,distorted or noisy averaged first-order image part from the first image.The first order image part may be removed by subtraction or otherequivalent mathematical operation.

It is to be appreciated that the methods and techniques described inthis invention can be applied to microscopy. Microscopy of biologicalsamples in particular can be degraded by the transmission and scatteringof light propagating through scattering and absorbing media that cansignificantly degrade the quality of the image. It is to be appreciatedthat substituting a microscope objective for a telescope as described incertain embodiment only alters the focal length of the optical systemand does not affect the image enhancement properties of this invention.

Another embodiment would entail generating enhanced images usingintensity products where more than two intensity measurements areavailable. This is especially useful for when the intensity deviationsdo not follow Gaussian statistics. This would involve simultaneousmeasurements of three or more sensors at a time. Our method would beapplied to generate enhanced images of the form <ΔI₁ ΔI₂ΔI₃>, <ΔI₁ ΔI₂ΔI₃ ΔI₄>, . . . , <ΔI₁ ΔI₂ . . . ΔI_(N)>. This has application to theinvestigation of turbulence, finding of non-classical photon behaviorand as a research tool exploring higher order correlation statistics,the investigation of the fundamental nature of quantum physics such asnon-local correlations, Bell inequalities, and EPR effects.

There is a trend in modern imaging devices, i.e. cameras, to providemore measured quantities at each pixel. Thus, the intensity measurementsmay include measurements such as wavelength (color or derived colormappings such as RGB, CMY, CMYK, etc.), polarization, Stokes parameters,spatial modes, orbital angular momentum (OAM), spin, etc. For example, acolor camera may provide separate wavelength measurements, red (R),green (G), and blue (B) intensity values at each pixel, the polarizationStokes parameters at each pixel, and modern infrared (IR) cameras canprovide measurements of long-wave infrared (LWIR) and mid-wave infrared(MWIR) intensities at each pixel of the imaging device, or combinationsof these measurements. In the current invention at least one of theavailable measured quantities is selected to provide the frame data forthe generation of the improved image of the region of interest.

It is to be appreciated that measurements of quantities such aswavelength and polarization are typically dependent on theresponsiveness of the measurement device to the quantity being measured.As an example, color cameras typically use band-pass filters arranged ina pattern over the pixels of the measurement device. These filters areusually labeled Red, Green, and Blue (R, G, and B). The wavelengths thateach of the R, G, G filters pass is centered at a particular wavelengthand also passes nearby wavelengths with wavelengths being more distantfrom the center wavelength being more highly attenuated. This effect isreferred to as the bandwidth of the filter. Polarization filters havesimilar bandwidths with respect to the orientation of the filter. Theresponsiveness to wavelength, polarization, etc., of an element on ameasurement may also be adjusted by applying, for example, larger orsmaller voltages to increase or decrease the degree to which eachelement (pixel) reacts to the wavelength or polarization of light thatinteracts with that pixel.

It is to be further appreciated that information that can be extractedfrom measurements made by at least one sensor may exist over a spectrumor bandwidth of time and space scales and that extraction of suchinformation may be facilitated by electronic, computational, and/oroptical filtering. For example, electronic filtering of measurements toemphasize lower or higher frequencies may enhance the observation ofcorrelations between sensor measurements and improve the generatedenhanced images. The character and amount of electronic and/or opticalfiltering needed to optimize enhanced images may vary with the types andmakeup of the sensors used, the physical properties of the area ofinterest, the physical properties of the source of illumination, thenature and physical properties of the intervening regions, (e.g.atmosphere, ocean, biological tissue, material, outer space, obscurants,etc) between the sensors, illuminators, and the region of interest. Byphysical properties we also mean to include physical, chemical,biological, and electronic properties.

Referring now to FIG. 1, in accordance with one preferred embodiment, inBox 1 a series of frames are inputted into the memory or input of aprocessor or image processor. The frame may be composed on a pluralityof pixels, typically in a two-dimensional (2D) array, that together forman image. Exemplary frames may be electronic image data such a TIFF orJPEG file. As used herein the terminology “processor” or “imageprocessor” as used in the following claims includes a computer,multiprocessor, CPU, minicomputer, microprocessor or any machine similarto a computer or processor which is capable of processing algorithms.The frames may comprise photographs of the same region of interest. Theregion of interest may be a scene, landscape, an object, a subject,person, or thing. In Box 2, the frame data or value of each pixel ateach pixel location is determined for a frame. In Box 3, the overallintensity of the frame is determined. The overall intensity correlatesto a “bucket value” determination in that overall intensity value doesnot comprise spatial information. Instead, it correlates to thesummation of the light intensity of a frame. In the case of a picture,the overall intensity correlates to the sum of the reflectedillumination. In the case of an electronic display formed by pixels, theoverall intensity is the summation each pixel value at each pixellocation within a given frame. At Box 4, the values in Box 2 aremultiplied by the value determined in Box 3. Box 5 represents the FrameData×Intensity Product for the frame. Inasmuch as the Frame Data is anarray of pixel values, the Frame Data×Intensity Product is also an arrayof values. At Box 6, the products of Box 5 (Frame Data×IntensityProduct) are repeated for each frame in a selected plurality of frames.As an example, one hundred frames may be selected. At Box 7, thesummation of the Frame Data×Intensity Products for the plurality offrames determined in Box 6 is divided by the number of frames (such asfor example one hundred) to determine the Frame Data×Intensity ProductAverage for the plurality of frames. As noted in Box 7, this ProductAverage is an array containing pixel values at each pixel locationwithin the frame.

FIG. 2 is a further description of a methodology of the presentinvention. Note that Box 7 is carried over from FIG. 1 into FIG. 2. InBox 8, the average frame data (or average value of each pixel at eachpixel location) is determined for the plurality of frames (e.g. 100) byaveraging the pixel values at each pixel location for the plurality offrames to determine an array of average pixel values. In Box 9, theaverage overall intensity for the plurality of frames is determined. Theis similar to the determination of Box 3 except that Box 3 is adetermination for a frame and Box 9 is an average for a plurality offrames. As stated with respect to Box 3, the overall frame intensitycorrelates to a “bucket value” determination in that overall intensityvalue does not comprise spatial information. Instead, it correlates tothe summation of the light intensity of a frame. In the case of apicture, the overall frame intensity correlates to the sum of thereflected illumination. In the case of an electronic display formed bypixels, the overall intensity is the summation each pixel value at eachpixel location within a given frame. The average overall intensity isthe summation of the values for a plurality of frames divided by thenumber of frames.

Box 10 represents the multiplication of Boxes 8 and 9 to form theAverage Frame Data×Average Intensity Product, which is an array. Asshown in the bottom portion of FIG. 2, the Average Frame Data×AverageIntensity Product is subtracted from the Frame Data×Intensity ProductAverage to form the refined image of Box 12.

It is postulated that the preferred methodology in effect subtracts outor negates the effects or errors due to the effects of turbulence or thelike. Most fluctuations caused by turbulence occur at the “edges” ofobjects. The algorithm focuses on the edges of letters, objects, etc. torefine the image edges.

FIG. 3 is a partial schematic block diagram illustration of the stepsfor performing an alternate preferred method of the present invention inwhich steps for dividing the frames into two sets are illustrated. InBox 1 a series of frames are inputted into the memory or input of aprocessor or image processor. The frames may comprise photographs of thesame region of interest. The region of interest may be a scene,landscape, an object, a subject, person, or thing. In Box 3, the overallintensity of the frame is determined. The overall intensity correlatesto a “bucket value” determination in that overall intensity value doesnot comprise spatial information. Instead, it correlates to thesummation of the light intensity of a frame. In the case of a picture,the overall intensity correlates to the sum of the reflectedillumination. In the case of an electronic display formed by pixels, theoverall intensity is the summation each pixel value at each pixellocation within a given frame. In Box 13, the average overall intensityfor all frames in the inputted (see Box1) is computed. To determine theaverage overall intensity, the summation of the intensities for framesis divided by the number of frames. In Box 14, the frames are separatedinto two sets; set one contains frames having an overall intensitygreater than the average overall intensity (derived in Box 13) and settwo contains frames having an overall intensity less than the averageoverall intensity (derived in Box 13)

FIG. 4 is a partial schematic block diagram illustration in which stepsperformed on the first set of frames are illustrated. The figure showshow to generate a first set of data which is referred to here at SET 1.SET 1 frame set includes frames having an overall intensity greater thanthe average overall intensity. The steps are comparable in effect to thesimilarly numbered frames in FIGS. 1 and 2, as denoted by the additionof a letter “A” suffix to the correlating element number. In Box 2A, theframe data or value of each pixel at each pixel location is determinedfor a frame. In Box 3A, the overall intensity (“bucket value”) of theframe is determined. In the case of a picture, the overall intensitycorrelates to the sum of the reflected illumination. In the case of anelectronic display formed by pixels, the overall intensity is thesummation each pixel value at each pixel location within a given frame.At Box 4, the values in Box 2A are multiplied by the value determined inBox 3A. Box 5A represents the Frame Data×Intensity Product for theframe. Inasmuch as the Frame Data is an array of pixel values, the FrameData×Intensity Product is also an array of values. At Box 6A, theproducts of Box 5A (Frame Data×Intensity Product) are repeated for eachframe in the first set of frames. At Box 7A, the summation of the FrameData×Intensity Products for the plurality of frames determined in Box 6Ais divided by the number of frames (such as for example one hundred) todetermine the Frame Data×Intensity Product Average for the first set offrames. As noted in Box 7A, this Product Average is an array containingpixel values at each pixel location within the frame.

In the lower portion of FIG. 4, note that Box 7A is repeated as shown bythe arrow. In Box 8A, the average frame data (or average value of eachpixel at each pixel location) is determined for the first set of framesby averaging the pixel values at each pixel location for the first setof frames to determine an array of average pixel values for the firstset. In Box 9A, the average overall intensity for the first set offrames is determined. This is similar to the determination of Box 3Aexcept that Box 3A is a determination for a frame and Box 9A is anaverage for a plurality of frames. As stated with respect to Box 3A, theoverall frame intensity correlates to a “bucket value” determination inthat overall intensity value does not comprise spatial information.Instead, it correlates to the summation of the light intensity of aframe. In the case of a picture, the overall frame intensity correlatesto the sum of the reflected illumination. In the case of an electronicdisplay formed by pixels, the overall intensity is the summation eachpixel value at each pixel location within a given frame. The averageoverall intensity is the summation of the values for a plurality offrames divided by the number of frames.

Box 10 represents the multiplication of Boxes 8A and 9A to form theAverage Frame Data×Average Intensity Product, which is an array. Asshown in the bottom portion of FIG. 4, the Average Frame Data×AverageIntensity Product is subtracted from the Frame Data×Intensity ProductAverage to form the refined image of Box 12A.

FIG. 5 is a partial schematic block diagram illustration of the stepsfor performing an alternate preferred method of the present invention inwhich steps performed on the second set of frames are illustrated. Thefigure shows how to generate a second set of data which is referred tohere at SET 2. The SET 2 frame set includes frames having an overallintensity less than the average overall intensity.

The steps are comparable in effect to the similarly numbered frames inFIGS. 1, 2, and 4 as denoted by the addition of a letter “B” suffix tothe correlating element number. In Box 2B, the frame data or value ofeach pixel at each pixel location is determined for a frame. In Box 3B,the overall intensity (“bucket value”) of the frame is determined. Inthe case of a picture, the overall intensity correlates to the sum ofthe reflected illumination. In the case of an electronic display formedby pixels, the overall intensity is the summation each pixel value ateach pixel location within a given frame. At Box 4, the values in Box 2Bare multiplied by the value determined in Box 3B. Box 5B represents theFrame Data×Intensity Product for the frame. Inasmuch as the Frame Datais an array of pixel values, the Frame Data×Intensity Product is also anarray of values. At Box 6B, the products of Box 5B (Frame Data×IntensityProduct) are repeated for each frame in a second set of frames. At Box7B, the summation of the Frame Data×Intensity Products for the pluralityof frames determined in Box 6B is divided by the number of frames (suchas for example one hundred) to determine the Frame Data×IntensityProduct Average for the second set of frames. As noted in Box 7B, thisProduct Average is an array containing pixel values at each pixellocation within the frame.

In the lower portion of FIG. 5, note that Box 7B is repeated as shown bythe arrow. In Box 8B, the average frame data (or average value of eachpixel at each pixel location) is determined for the first set of framesby averaging the pixel values at each pixel location for the first setof frames to determine an array of average pixel values for the firstset. In Box 9B, the average overall intensity for the second set offrames is determined. This is similar to the determination of Box 3Bexcept that Box 3B is a determination for a frame and Box 9B is anaverage for a plurality of frames. As stated with respect to Box 3B, theoverall frame intensity correlates to a “bucket value” determination inthat overall intensity value does not comprise spatial information.Instead, it correlates to the summation of the light intensity of aframe. In the case of a picture, the overall frame intensity correlatesto the sum of the reflected illumination. In the case of an electronicdisplay formed by pixels, the overall intensity is the summation eachpixel value at each pixel location within a given frame. The averageoverall intensity is the summation of the values for a plurality offrames divided by the number of frames.

Box 10 represents the multiplication of Boxes 8B and 9B to form theAverage Frame Data×Average Intensity Product, which is an array. Asshown in the bottom portion of FIG. 5, the Average Frame Data×AverageIntensity Product is subtracted from the Frame Data×Intensity ProductAverage to form the refined image of Box 12B.

FIG. 6 is a partial schematic block diagram illustration of the stepsfor performing an alternate preferred method of the present invention inwhich the refined image data for the second set is subtracted from therefined image data for the first set to form enhanced image data (Box12C).

Another alternate preferred method of the present invention applies theuse of techniques from the field of Compressive Imaging or CompressiveSensing. In this embodiment the “bucket” values for each frame of theseries is computed by integrating the values of the pixels within eachframe. This bucket data is stored for use per Eq. 5 below. The pixelvalues for each frame of the series are stored as a row in a matrix J.The improved image is computed by application of a Compressive Imaginginversion algorithm such as GPSR to solve Eq. 6. The improved image isreturned in the matrix R.

Virtual Ghost Imaging

Virtual Ghost Imaging refers to an imaging process which creates anenhanced image from a series of frames of an imaging subject based on aprocess related to Ghost Imaging.

Virtual Ghost Imaging in the current instance applies the followingprocess to a series of frames of an imaging subject. Inasmuch as theoverall frame intensity value determined in Box 3 correlates to the“bucket” value, a brief discussion of ghost imaging and reflective ghostimaging follows. Typically ghost imaging uses two detectors, one toobserve the light source and the other, single pixel or bucket detector,to observe the light scattering and reflecting from the target object.

G ⁽²⁾=<(I(x,y,t)_(source) I(t)_(bucket)>−<I(x,y,t)_(source)><(I(t)_(bucket)>  (1)

where < > denotes an ensemble average. If I_(source) and I_(bucket) arerecorded from the same target object, I_(bucket) may be computed as

I(t)=ƒdxdyI(x,y,t)_(source)  (2)

Basic Virtual Ghost Imaging

Results of an experiment conducted through turbulence using chaoticlaser or pseudo-thermal light are presented in FIG. 7. FIG. 8 shows thesame target computed with data taken using a typical two pathconfiguration.

FIG. 8 shows the average of the target images that were captured throughturbulence. Note the blurriness and lack of definition of the letters.As one can see the “ARL” in FIG. 7 is a visually better image than thatdisplayed FIG. 9. However, the “true” ghost image displayed in FIG. 8has better contrast and definition of the letters than FIG. 7 or FIG. 9.This is likely due to the use of a reference field that has not beenaltered by interaction with a target object.

Accelerated (Compressive Imaging) G⁽²⁾ Virtual Ghost Imaging

A relatively new mathematical field named Compressive Sensing(CS) orCompressive Imaging(CI) can be used to good effect within the context ofghost imaging. The first use of compressive techniques in the context ofGhost Imaging was performed by the Katz group (see O. Katz, et al.,“Compressive Ghost Imaging,” Appl. Phys. Lett., 95, 131110(2009))(hereby incorporated by reference) who demonstrated a ghost likeimaging proposal of Shapiro (see J. Shapiro, “Computational GhostImaging,” Phys. Rev. A 78 061802(R) (2008)). Their demonstration waslimited to a transmission object.

More recently the present inventors have performed experiments of thisnature using reflection objects.

The inventors' use of CS and CI is based on finding approximatesolutions to the integral equations using the GPSR mathematicalmethodology where

JR=B  (3)

and

R=R(x,y)  (4)

is the object reflectance. The term J is a matrix, where the rows arethe illumination patterns at time k and the B vector:

B=[B _(k)]  (5)

represents the bucket values. In cases where the system isunderdetermined (too few [B_(k)]), then L₁ constraints are applied tocomplete the system and sparseness is

$\mspace{20mu} {{\arg \; \min\limits_{\substack{\text{?} \\ \text{?}}}} = {{\underset{\text{?}}{\frac{1}{2}}{{B - {JR}}}_{2}^{2}} + {\tau {R}_{1}}}}$?indicates text missing or illegible when filed

used:The CS computational strategy takes advantage of the fact that it isnormally true in images that not all pixels in an image contain newinformation and the system is said to be sparse on some basis sincefewer degrees of freedom are needed to describe the system than thetotal number of pixels in the image. Data used to reconstruct an imagecan be referred to as sparse image data or sparse data. The parameter τis often a constant.

Compressive imaging (CI) results for the ARL target are presented usingEq. 2 and varying the τ parameter. FIG. 10 is an example of a resultwhere τ is too large and most of the pixel values are driven to 0. Onecan sense that the letters “ARL” are in the figure. Decreasing τ to avalue of 5e7, shown in FIG. 11 more portions of the “ARL” lettersappear. When τ is set to 2.5e7 the R is quite clear in FIG. 12 but theappearance of the A and the L are still degraded. Continuing with theexamination of the effect of the parameter τ, the value of τ is set to1e7. These results are shown in FIG. 13.

Finally, as a lower image quality bound, τ is set to equal 1e6. The“ARL” presented in FIG. 14 is quite degraded. These GPSR calculatedVirtual Ghost imaging results highlight the sensitivity of thecalculations to an external parameter (τ) which has no connection to theunderlying physics.

Air Force Resolution Target

Results were computed using Eq. 1 subject to the self-bucketing conceptof Eq. 2. These results are generated from a few hundred shots of theAir Force resolution target imaged at a 100 m distance throughturbulence.

A single image from this data set is presented in FIG. 15. This imageillustrates the distorting effect of turbulence on imaging. A simpleaveraging of 335 frames of the dataset that was performed is shown inFIG. 16. This average image has some better qualities that the singleframe image of FIG. 15 but one can still only resolve the coarser scalefeatures of the Air Force target.

Using the self-bucket ghost imaging concept on this dataset, an initialresult using only 2 frames of the dataset is displayed in FIG. 17A. Someof the edges in this image are very distinct and superior to areas ineither the instantaneous or the average images. When the entire datasetis used, as presented in FIG. 17B, the results are striking. Inparticular the 4 and 5 on the right side of the target are legible andthe small horizontal and vertical bars to the left of the numbers aredistinct; whereas those regions in the instantaneous and average imagesare simply blurs.

FIG. 18 depicts a high level block diagram of a general purpose computersuitable for use in performing the functions described herein, includingthe steps shown in the block diagrams, schematic representations, and/orflowcharts in the various embodiments disclosed throughout thisapplication. As depicted in FIG. 18, the system 50 includes a processorelement 52 (e.g., a CPU) for controlling the overall function of thesystem 50. Processor 52 operates in accordance with stored computerprogram code, which is stored in memory 54. Memory 54 represents anytype of computer readable medium and may include, for example, RAM, ROM,optical disk, magnetic disk, or a combination of these media. Theprocessor 52 executes the computer program code in memory 54 in order tocontrol the functioning of the system 50. Processor 52 is also connectedto network interface 55, which transmits and receives network datapackets. Also included are various input/output devices 56 (e.g.,storage devices, including but not limited to, a tape drive, a floppydrive, a hard disk drive or compact disk drive, a receiver, atransmitter, a speaker, a display, a speech synthesizer, an output port,and a user input device (such as a keyboard, a keypad, a mouse and thelike).

Further Improvement Embodiments

In a preferred embodiment there is provided a method to partition thevalues in the measured data sets, i.e. frames, into two or more groupsfor the frame data (reference fields) and overall frame intensities(bucket values). These groups are then used to compute products, orcross-correlations, between the different groupings. These individualproduct terms can be mathematically combined, via addition and/orsubtraction processes, to generate improve images of the target orscene. This method further adapts the techniques presented in the parentapplication. One key advantage to this method is that it is possiblewith the following embodiments to generate all positive valued imagesand largely eliminate background and noise effects. Other advantagesinclude the ability to operate on a computed partitioned image usingfunctions such as logarithms and exponentials to further increasecontrast or better identify objects and information on their properties.

The following embodiments are predicated on the appreciation that otheroperations involving the partitioned sets of above average and belowaverage measurements are beneficial to improve image quality in adverseconditions such as turbulence. These operations would include but arenot limited to cross-correlations between above average bucket (overallframe intensities) and below average reference fields. Typically fourcorrelation types are available when data is partitioned into twodistinct sets such as above the average and below the average values. Ina non-normalized form this can be written as

$\begin{matrix}{R_{m} = {\frac{1}{N_{M}}{\sum\limits_{1}^{N_{M}}\; {I_{a}I_{b}}}}} & (7)\end{matrix}$

where R_(m) is proportional to the correlation between the two data setsI_(a) and I_(b). N_(M) is the number of elements in the data sets beingconsidered. The subscript “m” indicates that the correlation R is aboutthe mean or average. Normalization is usually accomplished by dividingthe R_(m) by σ_(I) _(α) the standard deviation of I_(a) and σ_(I) _(b) ,the standard deviation of I_(b),

$\begin{matrix}{\mspace{79mu} {{RN}_{m} = {{\frac{R_{m}}{\text{?}\text{?}}.\text{?}}\text{indicates text missing or illegible when filed}}}} & (8)\end{matrix}$

The data sets I_(a) is the set of pixel measurements and I_(b) is theset of bucket measurements are used in the current invention as thedeviations from the average or mean.

I _(a)=(M _(a)−

M _(a)

)  (9a)

I _(b)=(M _(b)−

M _(b)

)  (9b)

where the M's indicate the measurement, either an individual pixel valueor the bucket value and the < > indicates and average over the ensembleof the measurements.

The product terms that comprise a particular R_(m) may be computedconditionally. The R_(m) can be called conditional product terms. Forinstance, R_(m) may be computed for the set of pixel values I_(a) thatare above the mean for those frames of data with bucket values I_(b)that are above the mean. For example:

$\begin{matrix}{R_{m}^{++} = {\frac{1}{N_{m}^{+}}{\sum\limits_{1}^{N_{m}^{+}}\; {I_{a}^{+}I_{b}^{+}}}}} & (10)\end{matrix}$

The other combinations of above/below mean pixel values and above/belowmean bucket values are computed similarly. These conditional R_(m) maythen be added or subtracted from each other to yield improved images ofa scene or target. Furthermore, it should be noted that the particularcombination of the R_(m) below

R _(m) ⁺⁺ +R _(m) ⁻⁻ −R _(m) ⁺⁻ −R _(m) ⁻⁺  (11)

is always positive.

Another alternative embodiment may include computing a G⁽²⁾ improvedimage. This improved image is then partitioned into pixels that are, forexample, above the spatial mean G⁽²⁾ and pixels that are below thespatial mean G⁽²⁾. These alternative positive/negative G⁽²⁾ partitionedimproved images can display higher contrast and can be further operatedupon by using mathematical operations such as logarithms to increase thedynamic range. It is to be recognized that other partitions are possibleto tailor results needed for specific applications.

Another alternative embodiment may include computing a R_(m) correlationimage by calculating the correlation coefficient between the I_(a) andI_(b) partitions where the I_(a) and I_(b) are not aligned in time orframe. For instance, at a particular pixel i,j there may be 10 frames inwhich that pixel is above the mean value of that pixel for all frames,and there may only be say 5 frames for which the I_(b) values is abovethe mean of I_(b). A correlation coefficient may be computed betweenthese two sets of data using:

$\begin{matrix}{{R\left( {i,j} \right)} = \frac{C\left( {i,j} \right)}{\sqrt{{C\left( {i,i} \right)}{C\left( {j,j} \right)}}}} & (12)\end{matrix}$

where R(i,j) is the correlation coefficient between variables i and j. Cindicates a covariance calculation between the variables i and j. Thedata sets are forced to be the same length, as required by the aboveR(i,j) process, by simply setting N to be the length of the data setwith fewer elements and then taking the first N values of the data setthat has a greater number of elements. This can lead to cross-timecorrelations that may prove useful for certain applications. The methodof calculating the values of R(i,j) may also use the processes outlinedabove.

Combinations of Conditional Product Terms

As an example, for imaging measurements of pixel values and bucketvalues that have been grouped into two sets each of above and belowtheir respective averages, there is a total of 80 possible ways topresent for output the results of the computed conditional productterms. For instance, each conditional product term may be presented fordisplay individually with either a positive or negative sign. Thus,individually for the four conditional product terms there is a total of8 ways to present them for display. Combinations of two conditionalproduct terms with signs allows for 24 options to present for display,combinations of three conditional product terms allows for 32, andcombinations of all four conditional product terms allows for 16 ways topresent for output and display.

Further Improvement Embodiments

FIGS. 19A and 19B are schematic block diagrams of alternate embodimentsthat provide a method to partition the values in the measured data sets,i.e. frames, into two or more groups for the frame data (referencefields) and overall frame intensities (bucket values). These groups arethen used to compute products, or cross-correlations, between thedifferent groupings. These individual product terms can bemathematically combined, via addition and/or subtraction processes, togenerate improve images of the target or scene. This method furtheradapts the techniques presented in the parent application. One keyadvantage to this method is that it is possible with the followingembodiments to generate all positive valued images and largely eliminatebackground and noise effects. Other advantages include the ability tooperate on a computed partitioned image using functions such aslogarithms and exponentials to further increase contrast or betteridentify objects and information on their properties.

FIG. 19A illustrates a diagram of alternate preferred embodiment 100,that generates enhanced averaged second-order images or movies of aregion of interest. Note, the region of interest may emit photons thatare not reflected or scattered photons from some other photon source. Aphoton source (labeled source) may include, for example, the sun, moon,coherent, incoherent, or partially coherent light, entangled photons,infrared radiation emitted by atoms and molecules, accelerating charges,lasers, light bulbs, light emitting diodes (LEDs), chaotic laser light,pseudo-thermal light generated by passing laser light through a rotatingground glass or other scattering material, stars, moons, clouds,planets, space objects, fluorescent lamps, electrical discharges,plasmas, bio-luminescence, and stimulated emission. The distant targetscene 101 comprises a target 102 which is an area used of the distanttarget scene for the particular region of interest developed for imagetesting. The element 103 represents optionally present turbulence orother obscuring or image degrading effects along photon path betweenlight source and target, and light source and imaging system or photonmeasurement devices or components. Light from the telescope 104(although other apparatus or direct viewing may be used withoutdeparting from the scope of the invention).is focused by a lens 106 andan image is formed within the camera 105. Telescope 104 may zoom in onor expand view of the selected region of interest. An image or photonmeasurement device 105 may be for example, a camera. Lens 106 couplesthe telescope 104 to image or photon measurement device 105. Theassembly 100 may optionally include a Barlow type lens for increasedmagnification of the region of interest. Control line 107 enablesreal-time control of data acquisition. Data transfer channel 108 extendsfrom image or photon measurement device 105 to the processor 110. Thecommunications channel 109 extends between processor 110 and dataacquisition and real-time control 126. The processor, memory, andalgorithms to generate enhanced average second-order images of theregion of interest are represented by the shaded area labeled 110.Representative frames 111 of image data are measured by measurementdevice 105. Virtual bucket detector 112 represents where the “bucket”values are computed from the measured frame data by spatially summingspecified pixel values. Software or hardware communications channel 113transfers bucket information between processor 110 or on the sameprocessor; Software or hardware communications channel 114 transfersbucket information between processors or on the same processor. Memoryportion 115 represents the computation and storage of the pixel averageimage and the average bucket value and computes pixel deviations of thepixel value from the average pixel value and computes bucket deviationsfrom the average bucket value. Image sets 118 and 119 representpartitions of spatially addressable pixel images into positive andnegative bucket deviation groups. Image memory portion or image set 119represents the grouping of negative value bucket deviation referenceimage frames. Image memory portion or image set 117 represents thegrouping of positive value bucket deviation reference image frames.Image memory portion or image set 118 represents the grouping ofpositive and negative valued pixel deviations for reference frames thathave positive valued bucket deviations. Image memory portion or imageset 119 represents the grouping of positive and negative valued pixeldeviations for reference frames that have negative valued bucketdeviations. Image memory portion or image set 120 represents thecollecting and averaging the product of negative value pixel deviationsand positive value bucket deviations for corresponding frames. Imagememory portion or image set 121 represents the; collecting and averagingthe product of positive value pixel deviations and positive value bucketdeviations for corresponding frames. Image memory portion or image set122 represents the collecting and averaging the product of negativevalue pixel deviations and negative value bucket deviations forcorresponding frames. Image memory portion or image set 123 representsthe collecting and averaging the product of positive value pixeldeviations and negative value bucket deviations for correspondingframes. Representation of an enhanced image 124 comprises addition,subtraction, normalization or other mathematical operations of thecomputed values of image sets 120, 121, 122, and 123. Block 125indicates some of the physics for the display of the enhanced averagesecond-order image of the region of interest. The block 125 is adepiction of the physics of a two-photon interference process wheresub-sources are at A and B respectively and detection is at B and Relements wherein the lines represent two alternate but indistinguishablepaths where a photon of sub-source A goes to detector R and a photon ofsub-source B goes to detector B or a photon from sub-source A goes todetector B and a photon of sub-source B goes to detector R to achieve ajoint detection. Block 126 comprises: data acquisition and real-timecontrol electronics.

FIG. 19B illustrates a diagram of alternate preferred embodiment 100,that generates enhanced averaged second-order images or movies of aregion of interest. Note, the region of interest may emit photons thatare not reflected or scattered photons from some other photon source. Aphoton source (labeled source) may include for example the sun, moon,coherent, incoherent, or partially coherent light, entangled photons,infrared radiation emitted by atoms and molecules, accelerating charges,lasers, light bulbs, light emitting diodes (LEDs), chaotic laser light,pseudo-thermal light generated by passing laser light through a rotatingground glass or other scattering material, stars, moons, clouds,planets, space objects, fluorescent lamps, electrical discharges,plasmas, bio-luminescence, and stimulated emission. The distant targetscene 101 comprises a target 102 which is an area used of the distanttarget scene for the particular region of interest developed for imagetesting. The element 103 represents optionally present turbulence orother obscuring or image degrading effects along photon path betweenlight source and target, and light source and imaging system or photonmeasurement devices or components. Light from the telescope 104(although other apparatus or direct viewing may be used withoutdeparting from the scope of the invention).is focused by a lens 106 andan image is formed within the camera 105. Telescope 104 may zoom in onor expand view of the selected region of interest. An image or photonmeasurement device 105 may be for example, a camera. Lens 106 couplesthe telescope 104 to image or photon measurement device 105. Theassembly 100 may optionally include a Barlow type lens for increasedmagnification of the region of interest. Beam splitter 127 operates tosplit the beam off to bucket detector 128 which is connected to channel129 which interconnects with image memory portion or set 115, whichrepresents the computation and storage of pixel image values. Thus,channel 129 operates to pass the measured bucket values to the processorfor computation of the average bucket value. Control line 107 enables ereal-time control of data acquisition. Data transfer channel 108 extendsfrom image or photon measurement device 105 to the processor 110. Thecommunications channel 109 extends between processor 110 and dataacquisition and real-time control 126. The processor, memory, andalgorithms to generate enhanced average second-order images of theregion of interest are represented by the shaded area labeled 110:Representative frames 111 of image data are measured by measurementdevice 105. Virtual bucket detector 112 represents where the “bucket”values are computed from the measured frame data by spatially summingspecified pixel values. Software or hardware communications channel 113transfers bucket information between processor 110 or on the sameprocessor; Software or hardware communications channel 114 transfersbucket information between processors or on the same processor. Memoryportion 115 represents the computation and storage of the pixel averageimage and the average bucket value and computes pixel deviations of thepixel value from the average pixel value and computes bucket deviationsfrom the average bucket value. Image sets 118 and 119 representpartitions of spatially addressable pixel images into positive andnegative bucket deviation groups. Image memory portion or image set 119represents the grouping of negative value bucket deviation referenceimage frames. Image memory portion or image set 117 represents thegrouping of positive value bucket deviation reference image frames.Image memory portion or image set 118 represents the grouping ofpositive and negative valued pixel deviations for reference frames thathave positive valued bucket deviations. Image memory portion or imageset 119 represents the grouping of positive and negative valued pixeldeviations for reference frames that have negative valued bucketdeviations. Image memory portion or image set 120 represents thecollecting and averaging the product of negative value pixel deviationsand positive value bucket deviations for corresponding frames. Imagememory portion or image set 121 represents the; collecting and averagingthe product of positive value pixel deviations and positive value bucketdeviations for corresponding frames. Image memory portion or image set122 represents the collecting and averaging the product of negativevalue pixel deviations and negative value bucket deviations forcorresponding frames. Image memory portion or image set 123 representsthe collecting and averaging the product of positive value pixeldeviations and negative value bucket deviations for correspondingframes. Representation of an enhanced image 124 comprises addition,subtraction, normalization or other mathematical operations of thecomputed values of image sets 120, 121, 122, and 123. Block 125indicates some of the physics for the display of the enhanced averagesecond-order image of the region of interest. The block 125 is adepiction of the physics of a two-photon interference process wheresub-sources are at A and B respectively and detection is at B and Relements wherein the lines represent two alternate but indistinguishablepaths where a photon of sub-source A goes to detector R and a photon ofsub-source B goes to detector B or a photon from sub-source A goes todetector B and a photon of sub-source B goes to detector R to achieve ajoint detection. Block 126 comprises: data acquisition and real-timecontrol electronics.

FIGS. 20A-20C illustrate a schematic block diagram of an alternatepreferred embodiment of the present invention wherein groups andsubgroups are used to determine product arrays to form an improvedimage. In: Box 301, a series of frames are entered into the memory orinput of a processor or image processor. The frames may comprisephotographs of the same region of interest. The region of interest maybe a scene, landscape, an object, a subject, person, or thing. In Box302 the pixel value for each pixel in each frame in the series of framesis determined. In Box 303, the overall intensity of the frame (orsummation of pixel values for each frame) is determined. The overallintensity correlates to a “bucket value” determination in that overallintensity value does not comprise spatial information. Instead, itcorrelates to the summation of the light intensity of a frame. In thecase of a picture, the overall intensity correlates to the sum of thereflected illumination. In the case of an electronic display formed bypixels, the overall intensity is the summation each pixel value at eachpixel location within a given frame. In Box 303, the average overallintensity for all inputted frames (see Box301) is computed. To determinethe average overall intensity, the summation of the intensities for allframes is divided by the number of frames. In Box 304 the overallintensity is determined, In Box 305 the overall intensity deviation isdetermined for each frame by subtracting the average overall intensityfrom the overall intensity for each frame. The overall intensitydeviation is a measure of the degree to which a frame is brighter ordarker than the Average Overall Intensity. In Box 306, an array ofaverage pixel values is formed (average value at each pixel location forthe series of frames.

Continuing in FIG. 20B, in Box 308P, if overall intensity deviation ispositive for a given frame, the frame is grouped into a first group P.In Box 309P, for each frame in the first group, select all pixels ineach frame with a positive deviation from the array of average pixelvalues and place in a first subgroup PP.

In Box 310P, for each frame in the first group, select all pixels ineach frame with a negative deviation from the array of average pixelvalues and place in a second subgroup PN. In Box 311P, for each frame inthe first group, select all pixels in each frame with a zero deviationfrom the array of average pixel values and place in a either the firstor second subgroup (PP or PN).

Continuing in FIG. 20C, in Box 308N, if the overall intensity deviationis negative for a given frame, the frame is grouped into a second groupN. In Box 309N, for each frame in the second group, select all pixels ineach frame with a positive deviation from the array of average pixelvalues and place in a third subgroup NP. In Box 310N, for each frame inthe second group, select all pixels in each frame with a negativedeviation from the array of average pixel values and place in a fourthsubgroup NN. In Box 311N, for each frame in the second group, select allpixels in each frame with a zero deviation from the array of averagepixel values and place in a either the first or second subgroup (NP orNN). In Box 312, for each pixel value for each frame in the eachsubgroup multiply each pixel value by overall intensity deviation forthat frame to obtain a first product array (or conditional productarray) for each frame in the subgroup (PN, PP, NP, NN). In Box 313, foreach subgroup, add up the product arrays in the subgroup and divide bythe number of frames in the subgroup to obtain a second or averageconditional product array for each subgroup. In Box 314, one of moresecond product arrays is selected to generate an image.

FIGS. 21-31 represent a partial schematic block diagram illustration ofthe steps for performing one embodiment of the present invention. InFIG. 21: Box 301, a series of frames are entered into the memory orinput of a processor or image processor. The frames may comprisephotographs of the same region of interest. The region of interest maybe a scene, landscape, an object, a subject, person, or thing. In Box303, the overall intensity of the frame is determined. The overallintensity correlates to a “bucket value” determination in that overallintensity value does not comprise spatial information. Instead, itcorrelates to the summation of the light intensity of a frame. In thecase of a picture, the overall intensity correlates to the sum of thereflected illumination. In the case of an electronic display formed bypixels, the overall intensity is the summation each pixel value at eachpixel location within a given frame. In Box 313, the average overallintensity for all frames in the inputted (see Box301) is computed. Todetermine the average overall intensity, the summation of theintensities for all frames is divided by the number of frames. In Box314 the overall intensity deviation for each frame is determined bysubtracting the Average Overall Intensity from the Overall Intensity foreach frame. The overall intensity deviation is a measure of the degreeto which a frame is brighter or darker than the Average OverallIntensity.

The method proceeds to FIG. 21. In Box 315, a series of frames areentered into the memory or input of a processor or image processor. Theframes may comprise photographs of the same region of interest. Theregion of interest may be a scene, landscape, an object, a subject,person, or thing. These frames are the same frames used in Box 301 andmay be reused if they are still available in the memory or input of theprocessor or image processor. In Box 316 the Average Frame Data isdetermined by computing the average value of each pixel at each pixellocation for the plurality of frames. In Box 317 the Deviation FrameData is determined by subtracting the Average Frame Data from the FrameData for all pixels in each frame for the plurality of frames.

The method proceeds via path 800 to FIG. 22 which shows how to generatea third set of data which is referred to here at SET 3. SET 3 dataincludes conditional product terms using frames having a positiveoverall intensity deviation and positive deviation frame data pixels.SET 3 may be determined as follows: In Box 318 frames with a PositiveOverall Intensity Deviation multiply the value of the Overall IntensityDeviation by the Positive Frame Data Deviation pixels within that set offrames. In Box 319 pixel locations (a), the square of the PositiveOverall Intensity Deviation (b), the product of the Positive OverallIntensity Deviation×the Positive Deviation Frame Data pixels and thesquare of the Positive Deviation Frame Data pixels are recorded andaccumulated. In Box 320 the pre-normalized Positive-Positive Productpixel values, 320(a), are determined by dividing the product of thePositive Overall Intensity Deviation×the Positive Deviation Frame DataFrame Data pixels by 319(a). The average of the squares of the PositiveOverall Intensity is determined by dividing 319(b) by 319 (a). Theaverage of the squares of the Positive Deviation Frame Data pixels isdetermined by dividing 319(d) by 319(a).

Continuing via path 900 to FIG. 28, the method proceeds to FIG. 23 InBox 321 the standard deviation of the Positive Overall IntensityDeviation is determined by taking the square root of the average of thesquares of the Positive Overall Intensity Deviation, 320(a). In Box 322the standard deviations of the Positive Deviation Frame Data pixels iscomputed by taking the square root of the average of the squares of thePositive Deviation Frame Data pixels, 320(b). Box 323 determines theNormalized Positive-Positive Product pixel values by dividing thepre-Normalized Positive-Positive Product, 320(a), by the product of thestandard deviation of the Positive Overall Intensity Deviation, 321, andthe standard deviation of the Positive Deviation Frame Data pixels, 322.

The method proceeds via path 800 to FIG. 24 which shows how to generatea fourth set of data which is referred to here at SET 4. SET 4 framedata includes conditional product terms using frames having a negativeoverall intensity deviation and negative deviation frame data pixels.SET 4 may be determined as follows: In Box 400 frames with a NegativeOverall Intensity Deviation multiply the value of the Overall IntensityDeviation by the Negative Frame Data Deviation pixels within that set offrames. In Box 401 pixel locations (a), the square of the NegativeOverall Intensity Deviation (b), the product of the Negative OverallIntensity Deviation×the Negative Deviation Frame Data pixels and thesquare of the Negative Deviation Frame Data pixels are recorded andaccumulated. In Box 402 the pre-normalized Negative-Negative Productpixel values, 402(a), are determined by dividing the product of theNegative Overall Intensity Deviation×the Negative Deviation Frame DataFrame Data pixels by 401(a). The average of the squares of the NegativeOverall Intensity is determined by dividing 401(b) by 401(a). Theaverage of the squares of the Negative Deviation Frame Data pixels isdetermined by dividing 401(d) by 401(a).

From Box 402 in FIG. 24, the method can proceed via path 901 to FIG. 25.In Box 403 the standard deviation of the Negative Overall IntensityDeviation is determined by taking the square root of the average of thesquares of the Negative Overall Intensity Deviation, 402(a). In Box 404the standard deviations of the Negative Deviation Frame Data pixels iscomputed by taking the square root of the average of the squares of theNegative Deviation Frame Data pixels, 402(b). Box 405 determines theNormalized Negative-Negative Product pixel values by dividing thepre-Normalized Negative-Negative Product, 402(a), by the product of thestandard deviation of the Negative Overall Intensity Deviation, 403, andthe standard deviation of the Negative Deviation Frame Data pixels, 404.At the conclusion of Box 405, the method proceeds via path 700 to FIG.30 for determining an improved image data 701, or to FIG. 31 fordetermining alternative improved image data 702 by an alternativeembodiment.

Returning to FIG. 22, the method also can concurrently proceeds via path800 to FIG. 26 which shows how to generate a fifth set of data which isreferred to here at SET 5. SET 5 frame data includes conditional productterms using frames having a negative overall intensity deviation andpositive deviation frame data pixels. SET 5 may be determined asfollows: In Box 800 frames with a Negative Overall Intensity Deviationmultiply the value of the Overall Intensity Deviation by the PositiveFrame Data Deviation pixels within that set of frames. In

Box 801 pixel locations (a), the square of the Negative OverallIntensity Deviation (b), the product of the Negative Overall IntensityDeviation×the Positive Deviation Frame Data pixels and the square of thePositive Deviation Frame Data pixels are recorded and accumulated. InBox 802 the pre-normalized Positive-Negative Product pixel values,802(a), are determined by dividing the product of the Negative OverallIntensity Deviation×the Positive Deviation Frame Data Frame Data pixelsby 801(a). The average of the squares of the Negative Overall Intensityis determined by dividing 801(b) by 801(a). The average of the squaresof the Positive Deviation Frame Data pixels is determined by dividing801(d) by 801(a).

From Box 802 in FIG. 26, the method can proceed via path 902 to FIG. 27.In Box 803 the standard deviation of the Negative Overall IntensityDeviation is determined by taking the square root of the average of thesquares of the Negative Overall Intensity Deviation, 802(a). In Box 804the standard deviations of the Positive Deviation Frame Data pixels iscomputed by taking the square root of the average of the squares of thePositive Deviation Frame Data pixels, 802(b). Box 805 determines theNormalized Positive-Negative Product pixel values by dividing thepre-Normalized Positive-Negative Product, 802(a), by the product of thestandard deviation of the Negative Overall Intensity Deviation, 803, andthe standard deviation of the Positive Deviation Frame Data pixels, 804.

At the conclusion of Box 805 in FIG. 27, the method proceeds via path700 to FIG. 30 for determining an improved image data 701, or to FIG. 31for determining alternative improved image data 702 by an alternativeembodiment.

Similar as discussed above with respect to the fifth set of data (SET5), returning to FIG. 22, the method also can concurrently proceed viapath 800 to FIG. 28 which shows how to generate a sixth set of datawhich is referred to here at SET 6. SET 6 frame data includesconditional product terms using frames having a positive overallintensity deviation and a negative deviation frame data pixels. SET 6may be determined as follows: In Box 600 frames with a Positive OverallIntensity Deviation multiply the value of the Overall IntensityDeviation by the Negative Frame Data Deviation pixels within that set offrames. In Box 601 pixel locations (a), the square of the PositiveOverall Intensity Deviation (b), the product of the Negative OverallIntensity Deviation×the Negative Deviation Frame Data pixels and thesquare of the Negative Deviation Frame Data pixels are recorded andaccumulated. In Box 602 the pre-normalized Negative-Positive Productpixel values, 602(a), are determined by dividing the product of thePositive Overall Intensity Deviation×the Negative Deviation Frame DataFrame Data pixels by 601(a). The average of the squares of the PositiveOverall Intensity is determined by dividing 601(b) by 601(a). Theaverage of the squares of the Negative Deviation Frame Data pixels isdetermined by dividing 601(d) by 601(a).

From Box 602 in FIG. 28, the method can proceed via path 903 to FIG. 28.In Box 603 the standard deviation of the Positive Overall IntensityDeviation is determined by taking the square root of the average of thesquares of the Positive Overall Intensity Deviation, 602(a). In Box 604the standard deviations of the Negative Deviation Frame Data pixels iscomputed by taking the square root of the average of the squares of theNegative Deviation Frame Data pixels, 602(b). Box 605 determines theNormalized Negative-Positive Product pixel values by dividing thepre-Normalized Negative-Negative Product, 602(a), by the product of thestandard deviation of the Positive Overall Intensity Deviation, 603, andthe standard deviation of the Negative Deviation Frame Data pixels, 604.

At the conclusion of Box 605 in FIG. 29, the method proceeds via path700 to FIG. 30 for determining an improved image data 701, or to FIG. 31for determining alternative improved image data 702 by an alternativeembodiment.

FIG. 30 is a partial schematic block diagram illustration of the stepsfor performing an alternate preferred method of the present invention inwhich the improved final image is determined by adding the above-mean,above mean image to the below-mean, below-mean images, subtracting theabove-mean, below-mean image, and subtracting the below-mean, above meanimage. Here, improved image data 701 is determined by adding “SET 3” 323to “SET 4” 405 and subtracting “SET 5” 505 and subtracting “SET 6” 605.

FIG. 31 is a partial schematic block diagram illustration of the stepsfor performing an alternate preferred method of the present invention inwhich the improved final image is determined by adding the above-mean,above mean image, the below-mean, below-mean image, the above-mean,below-mean image, and the below-mean, above mean image. Here, analternate embodiment of improved image data 702 is determined by adding“SET 3” 323, “SET 4” 405, “SET 5” 805 and “SET 6” 605 together.

FIG. 32 is a result computed using Eq. 1 subject to the self-bucketingconcept of Eq. 2. This result was from a few hundred shots taken of atypical scene using an infra-red camera. Some portions of the scene arepartially visible such as the lamp post in the middle and the clouds(32.a) in the sky background. Other features such as the safety bars(32.b) and the tree line in the background (32.c) are notdistinguishable.

FIG. 33 is a result computed using the same data from FIG. 32 butapplying the inventive procedure described in which the Rm conditionalproduct terms were combined with R_(m) ₊₊ +R_(m) ⁻⁻ −R_(m) ⁺⁻ −R_(m) ⁻⁺to produce the improved image. The superscripts indicate a positive ornegative deviation for the conditional product terms. In comparison withFIG. 32, the safety bars on the right of the image are now very distinct(33.b) and the edges of the clouds (33.a) are more sharply defined.Furthermore, in the distance a tree line (33.c) has been resolved by theapplications of the method described herein.

FIG. 34 is a result computed using the same data from FIG. 32 butapplying the inventive procedure described in which the R_(m)conditional product terms were combined with R_(m) ₊₊ +R_(m) ⁻⁻ +R_(m)⁺⁻ +R_(m) ⁻⁺ to produce the improved image. The superscripts indicate apositive or negative deviation for the conditional product terms. Incomparison with FIG. 32, the safety bars (34.b) on the right of theimage are distinct, the edges of the clouds (34.a) are more sharplydefined, and the distant tree line is resolved (34.c). A comparisonbetween the (a), (b), and (c) features of FIG. 33 shows some apparentrelative smoothing and a bit less contrast in FIG. 34.

FIG. 35 is a set of experimental results computed using a simple averageof the collected frame data (a); A baseline result computed using Eq. 1subject to the self bucketing concept of Eq. 2 (b); and two improvedimages when applying the partitioning procedure shown in FIG. 21 et seq.Both the Log positive (d) and negative (c) components of the base G⁽²⁾image show increased contrast and sharpening of edges. In particular, acomparison between (a1) on the average image and (c1) and (d1) of theimproved log negative and log positive images highlights that the number“6” is quite legible in the improved images. A comparison of the (b2),(c2) and (d2) regions highlights the increased contrast of the (c2) and(d2) area with respect to the baseline (b) image.

FIG. 36 presents improved image results using a few hundred shots takenof a typical scene using an infra-red camera. Some portions of the sceneare partially visible such as the lamp post in the middle and the cloudsin the sky background. The improved image was computed combining themethods shown in FIG. 30. Features such as the lamp post show much morecontrast and edge clarity. It should be noted that the process describedin FIG. 36 yields all positive results with background noise largelyeliminated.

Radiation Emitting Image Area

Objects within the field of view of a light sensing device typicallyreflect and scatter light from an external source of illumination aswell as emitting light. Depending on the source of illumination and thematerial composition of the objects within the scene the contributionsfrom reflected/scattered light and emitted light are often quitedifferent. Light that is reflected from an object typically polarizesthe reflected light while emitted light is often unpolarized. Emittedlight may be from physical processes that include, but are not limitedto, luminescence, fluorescence, “black body,” and thermal radiation.

Low Light Imaging

The present invention relates to a method that can be applied to improveimages of many types of low-light target objects and areas. Objects andtarget areas of interest are often very faint, either due, but notlimited to, to low photon emission rate, being very distant (radiationintensity with distance from a source follows an inverse square law,i.e. 1/(r²) where r is the distance from the center of the emitter tothe receiving device), the reflectance of the object or target area ofinterest being very small, the effective integration or shutter time ofthe measurement device being very short, the efficiency of the detectormay be small, or through attenuation and/or scattering due to the mediathrough which the radiation propagates. Low light conditions couldgenerally said to exist when the quality of the image degrades with thereduction of illuminating light or light received from the imaged areaof interest by the eye or a sensor such as a camera. A low signal tonoise ratio sensed by a sensor such as a CCD, CMOS, or single photondetector may also indicate low-light imaging conditions when the noiseof the sensor exceeds the measured light by the sensor. Outdoors betweendusk and dawn would typically be considered low-light conditions andindoors without bright overhead illumination may also produce low-lightconditions. In an environment when obstacles occlude the light sourcesuch as in a cave or thick forest also produce low-light conditions.Conditions that cause the use of intensity amplification are consideredlow-light conditions. FIG. 38 is an average image of a distant (2.33 km)target taken under low-light conditions. FIG. 37 demonstrates animproved enhanced image of the same target as shown in FIG. 38. A closeup picture of the target is shown in FIG. 39. It was determined that theconditions were low-light because the average image deteriorated to bevirtually free from recognizable patterns as the intensity of the sundecreased by more than a factor or two late in the afternoon.

Referring now to FIG. 40, in accordance with one preferred embodiment,in Box 1 a series of frames are inputted into the memory or input of aprocessor or image processor. The frame may be composed on a pluralityof pixels, typically in a two-dimensional (2D) array, that together forman image. Exemplary frames may be electronic image data such a TIFF orJPEG file. As used herein the terminology “processor” or “imageprocessor” as used in the following claims includes a computer,multiprocessor, CPU, GPU, minicomputer, microprocessor or any machinesimilar to a computer or processor which is capable of processingalgorithms. The frames may comprise photographs of the same region ofinterest. The region of interest may be a scene, landscape, an object, asubject, person, or thing. In Box 2, the frame data or value of eachpixel at each pixel location is determined for a frame. In Box 3Q, apredetermined number of pixels are selected, rather than the entire“overall intensity) as was the case in regard to the preferredembodiment of FIG. 1. The intensity value of the predetermined pixelscorrelates to a “bucket value” determination. It correlates to thesummation of the light intensity at the predetermined pixel locations.In the case of a picture, the overall intensity correlates to the sum ofthe reflected illumination at the predetermined pixel locations. In thecase of an electronic display formed by pixels, the intensity is thesummation each pixel value at each pixel location. At Box 4, the valuesin Box 2 are multiplied by the value determined in Box 3Q. Box 5Crepresents the frame data×sum of intensity of selected pixels productfor the frame, which will henceforth be referred to as the First ProductArray, which is also an array of values. At Box 6C, the sum of the FirstProduct Arrays for a plurality of frames (1-100) products is obtained.As an example, one hundred frames may be selected. At Box 7C, thesummation of the First Product Arrays determined in Box 6C is divided bythe number of frames (such as for example one hundred) to determine theAverage Of The First Product Arrays.

FIG. 41 is a further description of a methodology of an alternatepreferred embodiment of the present invention. Note that Box 7C iscarried over from FIG. 40 into FIG. 41. In Box 8, the average frame data(or average value of each pixel at each pixel location) is determinedfor the plurality of frames (e.g. 100) by averaging the pixel values ateach pixel location for the plurality of frames to determine an array ofaverage pixel values. In Box 9, the average overall intensity for theplurality of frames is determined. In the case of a picture, the overallframe intensity correlates to the sum of the reflected illumination. Inthe case of an electronic display formed by pixels, the overallintensity is the summation each pixel value at each pixel locationwithin a given frame. The average overall intensity is the summation ofthe values for a plurality of frames divided by the number of frames.

Box 10 represents the multiplication of Boxes 8 and 9 to form theAverage Frame Data×Average Intensity Product, which is an array. Asshown in the bottom portion of FIG. 41, the Average Frame Data×AverageIntensity Product is subtracted from the Average Of The First ProductArrays to form the refined image of Box 12.

Imaging Through Weather, Turbulence, and Obscuration

The methods and techniques described in conjunction with the presentinvention can be applied to improve and enhance imaging of subjectareas, objects, and scenes that would otherwise be degraded by theinfluence of scattering, particulates, bad weather conditions, includingbut not limited to rain or fog, turbulence, and obscuration, which maycreate at least one of the effects of artifacts, distortion, or noise inthe measured image data. The invention has been demonstrated to improveand enhance images as shown by FIGS. 37 through 39. Turbulence, a typeof obscuring property of the atmosphere, degrades the conventionalimage. In like manner, a conventional image degraded by bad or changingweather, low light, and/or obscuration would also be enhanced andimproved by the current invention.

Centriods or center-of-intensity calculations are commonly used in firstorder imaging applications such as fluorescence microscopy. Many imagesof an illuminated target are measured and the locations of fluorescingemitters are determined to greater resolution than the average of theimages by computing a center-of-intensity, or centroid, of the measuredfluorescence. Where the location of the emitting source is determined by

$R = {\frac{1}{I}{\int_{V}{{\rho (r)}r{V}}}}$

where I is the total intensity in the volume, V is the volume, r is aposition vector within the volume and ρ(r) is the value of the intensityat location r. However, two-photon “centroids” can be used to determinewith more resolution the location of the point of reflection orscattering of entangled photons pairs from a region of interest thatcause a measured coincidence detection event. In one embodiment one ofthe detectors may be a spatially resolving detector (D₁) and the otherdetector (D₂) is used as a non-spatially resolving “bucket detector.”When detectors D₁ and D₂ both make a measurement within some coincidencewindow T_(c) then a history of the locations on D₁ is recorded and acenter-of-coincidences can be determined and the location of the sourceof coincidences would be determined with sub-pixel resolution. In asecond embodiment where detector D₂ is also a spatially resolvingdetector then the spatial locations measured by each detector are usedto determine the center-of-coincidences. A third embodiment would changedetector D₁ to be the bucket measurement and detector D₂ to thespatially resolving detector. It is to be appreciated that inclusion ofmore detectors, i.e. D₃ . . . D_(N), would allow for higher-ordercoincidences to be measured and an N^(th) order center-of-coincidence tobe determined. These types of calculations can be performed when theentangled photons travel collinearly or non-collinearly.

Higher Order Partitiong

(1) Separation of different orders of interference with measurements atdifferent space-time locations for better contrast, visibility, anddifferent views of the physics. (2) With two levels, i.e. +/−, there are80 different views. (3) Separating the orders of photon interference asmanifest in images at the data +/− ensemble levels is useful for quantumimaging and quantum computing.

Stereoscoptic Images, Range, and Movies

Improved stereoscopic images and movies can be generated withapplication of the methods and techniques described in this invention. Atypical stereoscopic image or movie is produced by viewing a region ofinterest with two distinct and separated imaging sensors such as twocameras or two human eyes. The optical path from the region of interestto each sensor is slightly different and each sensor also emphasizesslightly different portions of the region of interest. Furthermore, thefield of view between each sensor and the region of interest istypically subject to different degrees of obscuration and degradingturbulence effects. The methods and techniques described in thisinvention can be applied to each of the stereoscopic channelsindividually, or in other embodiments cross-channel products between thereference fields and the buckets may be used, i.e. a right channelreference with a left channel bucket and a left channel reference with aright channel bucket. Range or depth maps of a region of interest can begenerated from stereographic images. Since the stereographic images areadversely affected by obscurants and turbulence the methods andtechniques described herein can be applied to generate improvedstereographic images which may then be used to generate more accuraterange or depth maps of the region of interest.

Ladar Per Pixel Range

Turbulence causes changes in the index of refraction in the atmosphere.These changes in the index of refraction cause a change in the speed oflight through media thereby causing ranging problems. For example, lighttravels slower thru air with higher index of refraction. Other factorsthat influence the index of refraction in the atmosphere include but arenot limited to temperature and concentrations of other materials such aswater. The current invention can be applied to mitigate the effects ofthe anisotropic, inhomogeneous, and temporally fluctuating changes inthe index of refraction along an optical path from a sender to areceiver. As an example a current type of LADAR system would direct alaser pulse to the region of interest, on sending the pulse the receiverdetector would start taking a series of measurements for N consecutivemeasurements or bins. Each of the measurements would correspond to a dTwhere light returning from the target area would be measured. Theparticular dT out of the N dT measurements that measured the greatestnumber of photons would be assumed to give the range to the region ofinterest (c M dT)/2 where c is the speed of light, M is the index of themeasurement time with the greatest number of photons measured, dT is thetime width of each bin. The factor of 2 corrects for the time of thepulse to travel from the laser to the target and back from the target tothe receiver. However, index of refraction variations generally spreadout the pulse sometimes to the extent where no single dT bin has thegreatest number of photon counts. Applying the methods and techniquesdescribed in this invention can mitigate this variable index ofrefraction induced degradation and generate improved depth or distanceto target maps of the region of interest.

Alternative Grouping for Reference and Bucket Field Measurements

An alternative embodiment of the current invention may use differentgroupings of pixels or spatially resolved measurements for the“reference” field and the “bucket” field measurements. As an example,the pixels of a CCD type sensor may be grouped into a checker board typepattern where the equivalent pixels to a red (even) or black (odd) boxon a checker board are treated as two distinct sensors. The even (odd)pixel set is then used as a set of “reference” measurements while theodd (even) pixel set is summed over and used as the set of “bucket”measurements. A G⁽²⁾ type image for each pairing, even/odd or odd/even,can then be computed using the methods and techniques described hereinto generate improved images of a region of interest. Each of thecomputed improved images may be examined separately or the can be addedor subtracted from each other to highlight different features of theregion of interest. It is to be appreciated that other groupings ofpixels or spatially resolved measurements would be advantageous forspecific applications or characteristics and details in the region ofinterest.

Color Images and Movies

Many current imaging devices are able to measure distinct portions ofthe electro-magnetic and optical spectrum. These devices such as colorCCD cameras, smart phone cameras, and video recorders often split thecolors of a region of interest into three color bands, i.e. red, green,and blue. Turbulence and obscurants typically impose an opticalfrequency dependent degradation on color images. The current inventioncan mitigate these degradations on a per color band basis. For example,using the methods and techniques described in this invention themeasured red, green, and blue color bands could be used to generate animproved red image, improved green image, and an improved blue image.These three improved color band images could then be collected into asingle improved “color” image. Other alternative embodiments wouldinclude using, for instance a “red” bucket with a “blue” referencefield. These types of cross-color calculations may be useful tohighlight or suppress desired features of the region of interest.

Extraction of Relative Dynamics Between Measurements Separated in Space,Time, or Both

Embodiments of the present invention can be used to determine thedynamics and relative dynamics contained in measurements of the regionof interest. The dynamics would consist of the information on thephysical, chemical, and biological processes as they evolve in space andtime. This would include but is not limited to the dynamics of lightsources, surface characteristics such as reflection, and scattering inthe transmission media.

Ghost Imaging with a Single Camera

Using ghost imaging we have shown that it is possible to generate animproved image of an area of interest using two detectors. The firstdetector is a sensitive light meter that measures light scattered orreflected from the area of interest. The second detector can be acamera, such as a charged coupled device (CCD), CMOS, or other devicecapable of imaging the relevant light source, such as the sun, stars,and moon outdoors, or light bulbs indoors, that illuminate the area ofinterest. In the case where the area of interest emits radiation such asinfrared radiation, detector one measures the combined reflected,scattered, and emitted radiation. Detector two measures the effects ofthe reflected, scattered, and emitted radiation sources. By combiningcoincident measurements of first detector and the second detector aghost image can be computed of the area of interest. It turns out thatthe ghost image can achieve higher resolution or greater clarity than animage of the area of interest using the camera type device alone takinga picture classically. This has been demonstrated in experiments andpublications of peer reviewed articles. An explanation for theimprovement of the image in turbulence is demonstrated by so calledtwo-photon models of coincident imaging found in our publication. Thereit is shown that turbulence aberrations cancel and do not affect theimproved resolution of the ghost image. Sometimes, a two-photon ghostimage is referred to as a second order image. Whereas, the camera imagetaken by classical means is referred to as a first order image.Classical theory and experiment show that first order images are smearedby turbulence.

It would be desirable to generate a second order ghost image with highresolution and clarity even through turbulence while using just a singlecamera which may or may not be a color camera. In the case of a colorcamera ghost imaging system and method can be applied to each color thatthe color camera measures a color ghost image will result. Also, thecamera could measure in infrared, UV, or more than one other wavelengthsand pseudo-coloring can be used to display the ghost image depicting theintensity in the different wavelength bands. One embodiment of how to dothis would be to treat the area of interest as a mirror or an imperfectmirror for the light sources. Thus, an array of pixel on the camera canimage the area of interest as a measure of the light sources just as thesecond detector above. The first detector which measures the reflectedand scattered light from the area of interest can be other or the samearray of pixels in the camera.

One method would be to use a black and white checkerboard pattern ofpixels where the white pixels act as detector two and the sum of themeasurements on the black pixels act as detector one. By combiningcoincident measurements of first detector and the second detector aghost image can be computed of the area of interest.

An alternate embodiment of this would be to place a beam splitterbetween the camera and the area of interest. The beam splitter can splitthe radiation into two parts, one part towards detector one and theother part towards detector two. For example with a 50/50 beam splitterhalf of the light from the area of interest would be directed to thepixels of the camera which act as the second detector. The remaininghalf of the light would be directed to a separate sensor which acts asthe first detector. The separate sensor could be attached to the cameraor even be a single pixel of the usual camera array. This secondembodiment is not as simple as the first embodiment. However, it may bemore accurate if the first detector has higher sensitivity or accuracy.By combining coincident measurements of first detector and the seconddetector a ghost image can be computed of the area of interest.

Another alternate embodiment would be to use all of the pixels of thecamera to act as detector two and the sum of the measurements of all ofthe pixels act as detector one. By combining coincident measurements offirst detector and the second detector a ghost image can be computed ofthe area of interest. Various combinations of the pixels including thosewhich overlap or do not overlap of the camera can used as detector oneand detector two. It has been demonstrated with experiments that thesecombinations described can be useful to obtain improved quality ghostimages.

In calculating the improved quality ghost image the processor creates acorrelation product of measurements from detector one times measurementsfrom detector two at coincident times for each pixel location measuredby detector two. Summing and averaging these correlations over anensemble of coincidence times yields a measure of the ghost image. Thisghost image is combined of both first order and second order imageinformation. Subtracting off the first order image which may be smearedby turbulence yields a second order ghost image which will be improvedover the first order image in that it will have higher resolution andless distortion due to turbulence. This image is often referred to as aG(2) image. When, for instance, thermal photons have Gaussian statisticsthen the G(2) image is expected to be positive. However, Gaussianstatistics may not accurately characterize the photon measurements madeby cameras imaging areas of interest. In addition, turbulence and otherphysical characteristics of the illuminating source, area of interest,camera, detectors one and two and intervening media may combine in waysto create photon measurements that would be best characterized asnon-Gaussian. Some of the physical characteristics that may contributeto the non-Gaussian photon statistics can include any or all ofinhomogeneous, non-stationary, anisotropic, non-ergodic, nonlinear, andquantum processes.

As it turns out empirically, G⁽²⁾ gives a good image of the area ofinterest. However, it is important to try to improve the G⁽²⁾ imagecontrast. Sometimes G⁽²⁾ contrast is not as high as ideally desirable.One way to improve the image contrast is to artificially scale the ghostimage by subtracting off any background. An analysis of the G⁽²⁾ signshows that when G⁽²⁾ is positive then the intensity deviations thatcomprise it are correlated. When G⁽²⁾ is zero then the intensitydeviations are uncorrelated. When G⁽²⁾ is negative then intensitydeviations are anti-correlated. When the negative parts of G⁽²⁾subtracts from the positive parts of G⁽²⁾ then an all positiveΔG⁽²⁾=G⁽²⁾ correlated−G⁽²⁾ anti-correlated can be computed. This methodcan be imaged by a display to have high contrast since the minimum iszero and other values are positive.

As described in the specification for U.S. patent application Ser. No.13/247,470 filed Sep. 28, 2011 (ARL 11-03) herein incorporated byreference, there are many possibilities to combine results from theconditional product terms, and R_(m) ⁺⁺, R_(m) ⁻⁻, R_(m) ⁺⁻, and R_(m)⁻⁺, computations to generate and display an improved image of the regionof interest. This sum, RI_(θx), is a weighed sum of correlated products.These correlated products would be formed from results of measurementsof quantum particles. There are many applications for the methoddescribed below including imaging using photons and other quantumparticles, measurement of gravitational waves, measurement of thequantum properties of sound, improved LIDARs, LADARs, improved medicalimaging, improved Earth surveillance from space, improved spacesurveillance from Earth or other places in space. It is helpful to putthe weighted sum of correlated products in a form where a user couldcompare the value of one vs. others for their application.

In particular, one useful way would be to weight terms in R_(m) ⁺⁺+R_(m)⁻⁻−R_(m) ⁺⁻−R_(m) ⁻⁺ by cos θ and sin θ factors, i.e. RI_(θ1)=cosθ(R_(m) ⁺⁺+R_(m) ⁻⁻)+sin θ(R_(m) ⁺⁻+R_(m) ⁻⁺),

where θ is an angle. This arrangement allows for continuously varyingthe contributions of the conditional product terms to display improvedimages to include an all positive image, an all negative image, a G⁽²⁾image and the negative of the G⁽²⁾ image.

Another alternative would apply a cos θ weight to the cross-termportion, RI_(θ2)=(R_(m) ⁺⁺+R_(m) ⁻⁻)+cos θ(R_(m) ⁺⁻+R_(m) ⁻⁺).

While the conditional product terms can be displayed independently as animage, the alternative means described here to present and display theresults would be useful for teaching and for detailed examination andanalysis of the properties of the region of interest.

In practice, a movie displaying RI_(θ1) or RI_(θ2) would be made thatscans through values of θ by the desired amount. For example, the moviecould start at θ=0 and each for each successive frame θ could beincremented by one degree until θ takes all values from 0 to 360degrees. This will show the continuous change of RI_(θ1) or RI_(θ2)exhibiting varying contrast, visibility, and resolution of the computedimage of the region of interest. For example when the symmetric termsare added to the asymmetric terms then a G⁽²⁾ image results when theasymmetric terms are subtracted from the symmetric terms then an allpositive image results. Which image is “better” for the user depends onthe needs of the user. For example, from RI_(θ2) one may obtain highcontrast for θ=180 degrees for an all positive image with highvisibility. In some cases, more resolution of the region of interest maybe found for θ=0 degrees.

Color cameras often have pixel sensors that are covered by colorfilters. That way light directed towards a pixel sensor first passesthrough a filter before interacting with the pixel sensor. The filtersare often laid out on a grid of pixel sensors in patterns that arevariations of the “Bayer” pattern. For example, the filters over eachpixel may be red, green, or blue arranged in a Bayer pattern. Usually50% of the pixels are green 25% are blue and 25% are red so that thepixels only respond to the color of light that transmits through theircolor filter. There is a process for converting a Bayer pattern image toa RGB image where a processor computes a RGB value for each pixel. Forexample at a pixel that ordinarily measures a “green” value the RGBvalues may be completed by interpolating surrounding red pixel valuesand surrounding blue pixel values to that pixel. One method to recordthe RGB image is a variation of the AVI standard. Video sequences orindividual pictures from a color camera are often recorded in AVI.Producing ghost images from AVI may result in high-quality ghost images.Occasionally, visual artifacts may be present in the ghost images due tothe Bayer to RGB interpolation and conversion process. For some imagesthat have been converted from Bayer to RGB it is possible to reverse theprocess to investigate what the ghost image would look like in theoriginal Bayer pattern format.

For example, one means to convert back to a Bayer pattern image from aRGB image would be to extract the Bayer pixel color for each pixellocation and zero out the interpolated color components. This wouldrecover the underlying Bayer pattern image which could then be used tocompute a ghost image that would be absent any artifacts from theinterpolative or Bayer-to-RGB process.

Resolution of Conditional Product Terms

This section contains information on the conditional product terms.Conditional product terms are terms formed by correlations betweenvalues measured above or below the mean value. The values may representintensity, polarization, or other physical quantities that are able tobe measured. When these quantities have quantum properties or are ableto be represented by quantum physics, such as photons or other quantumparticles, then the resolution and visibility of the conditional productterms may be enhanced beyond conventional limits.

Positive and Negative G⁽²⁾

The values of a particular G⁽²⁾ can be either positive or negative. Apositive G⁽²⁾ indicates that the measurements at an x, y pixel arecorrelated with the bucket measurements. A negative G⁽²⁾ indicates thatthe two measurements are anti-correlated.

CASE 1: As an example assume that an ensemble of measurements is made atlocation (a) and location (b). For this case assume that the ensembleconsists of two measurement realizations 1 and 2. For realization 1,I_(a)(1)=8 and I_(b) (1)=8. For realization 2, I_(a)(2)=2 andI_(b)(2)=2. G⁽²⁾=<I_(a)I_(b)>−<I_(a)><I_(b)>, where < > indicates andaverage over the ensemble of realizations. For this ensemble,G⁽²⁾=34−25=9. Examining Ia and Ib it is easy to see that the values ofboth of the measurements decrease at the same and typically referred toas correlated.

CASE 2: In a case where the ensemble of measurements is made at location(a) and location (b) for realizations 1 and 2, I_(a)(1)=2, I_(a)(1)=8and I_(b)(1)=8, I_(b)(2)=2. Then G⁽²⁾=16−25=−9. In this example I_(a)increases in magnitude from 2 to 8 while Ib decreases from 8 to 2. Thisis typically referred to as anti-correlated.

Multiple Buckets on Same Image

Bucket pixels can be allocated down to an individual pixel or a group ofpixels in particular frames. The pixels in a group of pixel selected tobe summed to a bucket (integrated intensities) value for a frame neednot be contiguous. The bucket pixel or pixels may also be chosen on aper-frame basis. For instance, all pixels within 10% of the maximumpixel value for a frame may be chosen to be summed over for the bucketvalue for that frame. The spatial distribution of those pixels within10% of the maximum pixel value is likely to change from frame to framebased on lighting conditions and turbulence/obscuration effects betweenthe target area and the measurement system. Furthermore, single pixelbuckets or multiple pixel buckets may be utilized.

Bucket Pixel Selection for Each Reference Pixel

The criteria for selecting a set of pixels to use as a “bucket” valueincludes but is not limited to finding those pixels which contribute toimproved contrast of the refined image, improving the convergence rateof the refined image, improving resolution of the refined image,improving the refined image fidelity to the imaged scene or target,finding optimal placement/selection of pixel sensors as “buckets” tominimize noise in refined image, providing enhanced resolution, tocomply with certain G⁽²⁾ mathematical constraints such as positivity,and to explore the computational, mathematical, and/or physical effectsof changing the bucket configuration. For example, we can compute a G⁽²⁾using all the pixels in a frame to represent the bucket. A G⁽²⁾ refinedimage may also be computed using only one pixel out of a set of pixelsin a frame. We may see from comparison of two G⁽²⁾ images that the G⁽²⁾image with the many pixel bucket provides different images than a G⁽²⁾image using a single pixel. Furthermore, the G⁽²⁾ image changes with thepixel location or other physical property such as wavelength in theensemble of frames array. This comparison provides image informationabout the effect of a single pixel “bucket,” its location and the manypixel “bucket” average of the ensemble of such pixels. From thisinformation, the effect of bucket size (number of pixels used for thebucket) and shape (distribution of pixels in space and time used as thebucket) on the properties of the refined image can be determined whichprovides theoretical and practical benefits such as determining andimplementation of optimized bucket selection to provide enhanced imagesof the target or region of interest.

Each pixel within a set of measurements (I), typically intensity values,taken using a camera may be indexed by a location x_(i), y_(i), k, i.e.I(xi, yj, k). When the locations of the measurement elements of thesensor are not arbitrarily distributed a pixel value can be indexed asI(i, j, k). The values of i vary from 1 to IMAX where IMAX is the numberof horizontal pixels on the sensor, j varies from 1 to JMAX where JMAXis the number of vertical pixels on the sensor and k is the identifierfor the frame number which varies from 1 to NFRAMES. For example asensor may have 8 pixels in the horizontal (IMAX=8) and 8 pixels in thevertical (JMAX=8). If 10 images where measured with this sensor thenNFRAMES would equal 10. In the current invention, a “bucket” value B(i,j, k) is determined on a per pixel basis for each pixel in the set ofmeasured values. The “bucket” for say, pixel i=2, j=6, k=3 could be anormalized summation of pixels values. For example B(2, 6, 3) could be[I(1, 1, 1)+I(3, 8, 7)]/2. B(3, 1, 1)=[I(2, 1, 2)+I(1, 9,3)+I(10,10,9)]/3. Then these bucket values B(i, j, k) can be used tocompute a G⁽²⁾ image as <I(i, j, *B(i, j, k)>−I(i, j)>*<B(i, j)> where< > indicates an ensemble average over the number of frame. In thecurrent invention a previous embodiment would have the “bucket” or“overall frame intensity” defined as B(k)=ΣI(i, j, k) where thesummation is over all i=1 to IMAX and j=1 to JMAX. The B(k) value forthis embodiment would be applied to compute a G⁽²⁾ images as G⁽²⁾=<I(i,j, k)*B(k)>−<I(i, j)>*<B>.

Correlations Between Different Wavelengths

Reference values and bucket (integrated intensities of pixels) valuesmay be allocated based upon the wavelength, or color, of the measuredphotons. For instance, measured green values on a color CCD or CMOScamera may be used to provide the reference field pixel measurements andmeasured red values may be allocated or summed to provide the value ofthe bucket measurement for a frame.

Infrared Between Bands

Modern advanced infrared cameras may provide per-pixel co-locatedmeasurements of infrared wavelengths in, for instance, the mid-waveinfrared (MWIR) and long-wave infrared (LWIR) wavelength bands. One bandmay be used to provide reference pixel values and pixel in the otherband can be allocated or summed to provide bucket values. A further,local G⁽²⁾ type calculation may take place wherein deviations from theensemble mean of one wavelength band can be multiplied with deviationsfrom the ensemble mean of the other wavelength band. These deviationproducts are performed for each pixel in a frame and summed over all theframes of data in the ensemble. A preferred methodology may thereforecomprise dividing this deviation product sum by the total number offrames in the ensemble to yield a G⁽²⁾ image of the area of interest.

Coordinate Shifting of Pixel Values

FIG. 42 is a schematic diagram of an alternate preferred embodimentinvolving coordinate shifting of pixel values. As illustratedschematically in FIG. 42, element 930 is a frame of pixel values, suchas from a charge coupled device (CCD). Note that only a small,representative fraction of the pixel values in an entire frame isdepicted. A charge-coupled device (CCD) as used herein is a device forthe movement of electrical charge, usually from within the device to anarea where the charge can be manipulated, for example conversion into adigital value. Element 931 represents the shifted set of pixels valuesfrom element 930 where the pixel values of 930 are shifted to the leftby 1 column; the furthest left column is filled with 0 values. Box 932represents the performance of the inventive calculations with theshifted pixel values including the calculations for ensemble averagearrays of pixel values, ensemble average arrays of shifted pixel values;deviations of pixel value arrays and shifted pixel value arrays fromtheir respective averages, and calculation of the enhanced final image;ie., the G⁽²⁾ Image for process for products of all P_(i,j)×I_(i,j). Box933 is the explanation for the G⁽²⁾ Image for<P_(i,j)×I_(i,j)>−<−P_(i,j)>x<I_(i,j)> that is represented by Box 934.930 Sample array of pixel values from CCD or other imaging device. Thecalculated enhanced final image pixel values may be transferred from Box934 to memory for storage or to a display for viewing.

FIG. 43 is a partial schematic block diagram illustration of the stepsfor performing an alternate preferred method involving coordinateshifting of pixel values

FIG. 44 is a schematic block diagram illustration of the steps forperforming an alternate preferred method involving coordinate shiftingof pixel values. Taken together, FIGS. 43 and 44 outline the steps of analternate preferred methodology involving coordinate shifting of pixelvalues. Boxes which are not labeled with a suffix “S” are the same asdescribed with respect to FIGS. 1, 2, 40 and 41. Box 38 representsdetermining shifted frame data (value of each pixel at shifted pixellocation) for each frame. Box 58 represents the frame data x shiftedframe data product. Box 68 is the sum of the frame data×shifted framedata products for a plurality of frames (1-100). Box 78 represents thedivision by the number of frames (e.g. 100) to obtain frame data×shiftedframe data product average [contains pixel values at each pixellocation]. Box 98 represents the determining of the average shiftedframe data (average of each shifted pixel at each pixel location for theplurality of frames). Box 118 represents the average frame data×averageshifted frame data product, which is subtracted from Box 78 to producethe enhanced image.

FIGS. 45 to 47 present schematic diagrams for methods and systemembodiments of calculating enhanced images of a region of interestwherein the values used to multiply the pixel values of the “firstarray” are generalized to be a selected set of pixel values within theplurality of frames. The sum of selected intensity values is determinedfor each pixel in the “first array.” Effectively this is a per pixel“bucket value” that would allow for the tailoring of the “bucket” valuesto meet imaging requirements. As an example, pixels in the first arraythat are not representative of the region of interest can be excluded.Such pixels may include those pixels on the sensor that are returningmeasurement values of 0 or pixels that are returning values that aremuch larger in magnitude than nearby pixels.

FIG. 45 is a partial schematic block diagram illustration of the stepsfor performing an alternate preferred method involving a generalizedmeans to select pixels to use as “bucket” values for each pixel in theplurality of frames of data. FIG. 46 is a schematic block diagramillustration of the steps for performing an alternate preferred methodinvolving selecting a set of pixels to generate a normalized sum to useas a “bucket” value for each measured pixel value in the plurality offrames.

Taken together, FIGS. 45 and 46 outline the steps of an alternatepreferred methodology involving selecting a set of pixels to generate anormalized sum to use as a “bucket” value for each measured pixel valuein the plurality of frames. Boxes which are not labeled with a suffix“S”, “T”, or “U” are the same as described with respect to FIGS. 1, 2,40, 41, 43 and 44. Box 3T represents determining the selecting sets ofpixels to generate a normalized sum to use as a bucket values for eachpixel for each frame. Box 3U represents normalizing the sum of theselected sets of pixels by dividing the sum of the pixel values in eachset of pixel by the number of pixels within each set of selected pixesl.Box 5T represents the frame data×normalized pixel intensity sum product.Box 6T is the sum of the frame data×normalized pixel intensity sumproducts for a plurality of frames (1-100). Box 7T represents thedivision by the number of frames (e.g. 100) to obtain frame data×snormalized pixel intensity sum product average [contains pixel values ateach pixel location]. Box 9T represents the determining of the averagenormalized pixel intensity sum (average of each normalized pixelintensity sum at each pixel location for the plurality of frames). Box11T represents the average frame data×average normalized pixel intensitysum product, which is subtracted from Box 7T to produce the enhancedimage.

FIG. 47 presents a block diagram of a system for image improvement 2000including a processor 2001, memory 2002, input channel 2003, display2004, and an image storage device 2007. The system 2000 is operative toexecute the image improvement algorithm 2006 on processor 2001 whereinthe algorithmic program and input image data frames are stored in memory2002. The steps of the algorithmic process 2006 are detailed as follows:In Box 2008 a series of frames. e.g. 100 frames, are inputted into thememory or input of a processor or image processor. The frame may becomposed on a plurality of pixels, typically in a two-dimensional (2D)array, that together form an image. Exemplary frames may be electronicimage data such a TIFF or JPEG file. As used herein the terminology“processor” or “image processor” as used in the following claimsincludes a computer, multiprocessor, CPU, minicomputer, microprocessoror any machine similar to a computer or processor which is capable ofprocessing algorithms. The frames may comprise photographs of the sameregion of interest. The region of interest may be a scene, landscape, anobject, a subject, person, or thing. In Box 2009, the frame data orvalue of each pixel at each pixel location is determined for each frame.In Box 2010, a predetermined number of pixels are selected for eachpixel in the plurality of frames, rather than the entire “overallintensity) as was the case in regard to the preferred embodiment of FIG.1., to sum and use as the “bucket value” for the corresponding pixel. InBox 2011 The intensity value of the predetermined pixels correlates to a“bucket value” determination. It correlates to the summation of thelight intensity at the predetermined pixel locations. In the case of apicture, the overall intensity correlates to the sum of the reflectedillumination at the predetermined pixel locations. In the case of anelectronic display formed by pixels, the intensity is the summation eachpixel value at each pixel location. Box 2011 a divides the summation ofpixel values by the number of pixels for each set of pixels selected. AtBox 2012, the values in Box 2009 are multiplied by the values determinedin Box 2011 a. Box 2012A represents the frame data×normalized sum ofintensity of selected pixels product for the frame, which willhenceforth be referred to as the First Product Array, which is also anarray of values. At Box 2013, the sum of the First Product Arrays for aplurality of frames (1-100) products is obtained. As an example, onehundred frames may be selected. At Box 2014, the summation of the FirstProduct Arrays determined in Box 2013 is divided by the number of frames(such as for example one hundred) to determine the Average Of The FirstProduct Arrays. In Box 2016, the average frame data (or average value ofeach pixel at each pixel location) is determined for the plurality offrames (e.g. 100) by averaging the pixel values at each pixel locationfor the plurality of frames to determine an array of average pixelvalues. In Box 2017, the averaged normalized pixel intensity sums foreach pixel in the plurality of frames is determined. In the case of apicture, the overall frame intensity correlates to the sum of thereflected illumination. In the case of an electronic display formed bypixels, the overall intensity is the summation each pixel value at eachpixel location within a given frame. The average overall intensity isthe summation of the values for a plurality of frames divided by thenumber of frames. Box 2018 represents the multiplication of Boxes 2016and 2017 to form the Average Frame Data×Average Normalized PixelIntensity Sum, which is an array. The Average Frame Data×AverageNormalized Pixel Intensity Sum is subtracted from the Average Of TheFirst Product Arrays to form the refined image of Box 2021. In oneexemplary configuration of system 2000, the processor 2001 may be a DellPrecision T7500 Xeon 64 bit CPU, the memory 2002 be 12 Gigabytes of DDR3RAM, the input channel 2003 may be a serial ATA (SATA), and the imagestorage 2007 may be a hard disk drive.

FIG. 48 illustrates a high level block diagram that generates enhancedaveraged second-order images or movies of a region of interest. Note,the region of interest may emit photons that are not reflected orscattered photons from some other photon source. A photon source mayinclude for example the sun, moon, coherent, incoherent, or partiallycoherent light, entangled photons, infrared radiation emitted by atomsand molecules, accelerating charges, lasers, light bulbs, light emittingdiodes (LEDs), chaotic laser light, pseudo-thermal light generated bypassing laser light through a rotating ground glass or other scatteringmaterial, stars, moons, clouds, planets, space objects, fluorescentlamps, electrical discharges, plasmas, bio-luminescence, and stimulatedemission. The distant target scene 3101 comprises a target 3102 which isan area used of the distant target scene for the particular region ofinterest developed for image testing. The element 3103 representsoptionally present turbulence or other obscuring or image degradingeffects along photon path between light source and target, and lightsource and imaging system or photon measurement devices or components.Light from the telescope 3104 (although other apparatus or directviewing may be used without departing from the scope of the invention)is focused by a lens 3106 and an image is formed within the camera 3105.Telescope 3104 may be configured to zoom in on or expand view of theselected region of interest, as well as zoom out. An image or photonmeasurement device 3105 may be for example, a camera. Lens 3106 couplesthe telescope 3104 to image or photon measurement device 3105. Theassembly may optionally include a Barlow type lens for increasedmagnification of the region of interest. Control line 3107 enables ereal-time control of data acquisition. Data transfer channel 108 extendsfrom image or photon measurement device 3105 to the processor 3110. Thecommunications channel 3109 extends between processor 3110 and dataacquisition and real-time control 3126. The processor, memory, andalgorithms to generate enhanced average second-order images of theregion of interest are represented by box 3110. Box 3111 is a memoryassociated with processor 3110 to store input images, algorithms,intermediate computations, and enhanced second order images of theregion of interest. Box 3112 represents software operationallyconfigured to perform the inventive process described in FIGS. 40-41,and/or FIGS. 45-46. Box 3113 is a display operationally connected bychannel 3114 to processor 3110 to display the generated enhancesecond-order image of the region of interest. 3114 indicates a cable orcommunications channel to connect display 3113 to processor 3110 totransfer the generated enhanced second-order images of the region ofinterest to the display. Block 3126 comprises the data acquisition andreal-time control electronics.

FIGS. 49 to 51 present block system diagrams for systems that provideillumination of a distant area of interest for use in generating anenhanced image of the target or area of interest. The illumination canbe entangled photons, incoherent photons, or coherent photons.Illumination has benefits for instances when the environmentalillumination is insufficient or for instanced when it is desirable touse particular types of photons, e.g. entangled photons for rangedetermination or absorption characteristics of the area of interest,etc., coherent photons of a particular wavelength, or broad-bandincoherent light to meet particular application requirements.

FIG. 49 presents a block diagram of an embodiment of a system for imageimprovement.

Box 4901 indicates a distant target scene area. Box 4902 indicates thetarget which is an area of the distant target scene for the particularregion of interest selected for improvement and enhancement. Box 4903indicates optionally present turbulence or other obscuring or imagedegrading effects along photon path between light source and target, andlight source and imaging system or photon measurement devices orcomponents. Box 4904 is a telescope. Telescope 4904 (although otherapparatus or direct viewing may be used without departing from the scopeof the invention). Telescope 4904 may zoom in on or expand view of theselected region of interest, as well as zoom out. Telescope 4904 is usedto transmit entangled photon pairs to the region of interest and receiveentangled photon pairs reflected or scattered from the region ofinterest. Box 4905 is an optical circulator: An optical circulator is anoptical element that transfers incoming photons to the next port of thecirculator unlike a beam-splitter where photon paths are split to two ormore ports. Box 4906 is an entangled photon source. The entangled photonsource generates entangled photon pair that are entangled intime-energy, H-V polarization or between other conjugate pair propertiesof the photons. Exemplary examples of entangled photo-sources includeentangled photons generated via Spontaneous Parametric Down-conversion(SPDC) in a nonlinear crystal such a Beta-Barium Borate (BBO) orpotassium titanyl phosphate (KTP), entangled photons generated in aquasi-phase matched nonlinear media such as periodically poled KPT(PPKTP) or periodically poled Lithium Niobate (PPLN), and entangledphotons generated in a four-wave mixing process in a nonlinear opticalfiber.

Box 4907 is a polarizing beamsplitter, dichroic-mirror or other opticalelement that operates to direct one portion of an entangled photon pairtowards spatially resolving detector 1 and directs the remaining portionof an entangled photon pair toward spatially resolving detector 2. Theportions of the entangled photon that are directed towards, for example,detectors 4909 and 4910 are selected by the element in Box 4907. As anexample, an entangled photon produced in an entangled polarization state|HH>|VV>+|VV>|HH> may be generated with frequencies ν₁ and ν₂; thefrequencies are not correlated with the polarizations. Then, Box 4907could be configured with a dichroic mirror that would operate to direct,for example, ν₁ towards detector 4909 and ν₂ towards detector 4910 formeasurement. The choice of the element for Box 4907 of course woulddepend on the configuration of the system and/or the type ofentanglement being used to illuminate the target.

Element 4908 is a lens used to focus the photons onto detector 1 anddetector 2. Box 4909 indicates spatially resolving detector 1. Spatiallyresolving detector 1 measures the time and spatial (x, y) location ofone part of an entangled pair that has interacted with the remote scene,target or subject. Box 4910 indicates spatially resolving detector 2.Spatially resolving detector 2 measures time and spatial (x, y) locationof the second part of an entangled pair that has interacted with theremote scene, target or subject. Detectors placed at multiple, diverselocations allow imaging from separate vantage points. One can alsogenerate stereo views from different locations and composite imaging toprovide a large field of view, increased depth, texture, and resolutionof the region of interest. Furthermore, the measurements from separatesensors can be utilized for higher-order images of the region ofinterest.

Volumetric Ghost Imaging

A volumetric ghost image of a region of interest would be able toprovide a 3D space filling representation of the region of interest thatwould not be degraded by the adverse effects of obscurants orturbulence. The 3D space filling representation could then be output toa 3D computer rendering system, 3D printer, orcomputer-numerical-control (CNC) device for visualization or for thegeneration of a physical model of the region of interest that wasvolumetrically ghost imaged. One could have a series of such volumetricimages to form a 3D movie or a moving representation of the 3D volume.

Some exemplary detectors which may be used for the detector 1 anddetector 2 include charge coupled devices (CCD), CMOS, SPAD arrays,quantum well, LIDAR, LADAR, video device, spatial sensor, light field(plenoptic) camera, gyro-stabilized camera, spatial phase sensitivecamera, or range sensor.

Box 4911 indicates coincidence and timing electronics that operates toregister when a pixel on detector 1 and a pixel on detector 2 occurinside within a user defined coincidence window ΔT_(c). A coincidencewindow is a time difference within which two photon measurements aredefined to be co-incident. The timing electronics further operate torecord the time that has elapsed since a user chosen laser pulse and thefirst coincidence pair detection for ranging calculations. Box 4912indicates a processor, memory, and algorithms to generate enhancedaverage second-order images of the region of interest. Box 4913indicates memory associated with processor 4912 to store input images,algorithms, intermediate computations, and enhanced second order imagesof the region of interest. Box 4914 indicates software operationallyconfigured to perform the image improvement and enhancement processesTheprocessing corresponding to Box 4914 may include at least one of theinventive methods described, for example by FIGS. 40-41, to compute asecond or higher order improved image of the target area. Box 4915 is adisplay operationally connected to processor 4912 to display thegenerated enhanced second-order or higher order image of the region ofinterest.

FIG. 50 presents a block diagram of an embodiment of a system for imageimprovement. Box 4901 indicates a distant target scene area. Box 4902indicates the target which is an area of the distant target scene forthe particular region of interest selected for improvement andenhancement. Box 4903 indicates optionally present turbulence or otherobscuring or image degrading effects along photon path between lightsource and target, and light source and imaging system or photonmeasurement devices or components. Box 4904A is a first telescope(although other apparatus or direct viewing may be used withoutdeparting from the scope of the invention). Telescope 4904A may zoom inon or expand view of the selected region of interest, as well as zoomout. Used to transmit entangled photon pairs to the region of interest.Box 4904B is a second telescope (although other apparatus or directviewing may be used without departing from the scope of the invention).Telescope 4904B may zoom in on or expand view of the selected region ofinterest. 4904B is configured to receive entangled photon pairsreflected or scattered from the region of interest. By providingseparate first and second telescopes, it is possible to provide betterresults and independently control light transmitted and received.

Box 4906 is an entangled photon source. The entangled photon sourcegenerates entangled photon pair that are entangled in time-energy, H-Vpolarization or between other conjugate pair properties of the photons.Box 4907 is a polarizing beamsplitter, dichroic-mirror or other opticalelement that operates to direct one portion of an entangled photon pairtowards spatially resolving detector 1 and directs the remaining portionof an entangled photon pair toward spatially resolving detector 2.Element 4908 is a lens used to focus the photons onto detector 1 anddetector 2. Box 4909 indicates spatially resolving detector 1. Spatiallyresolving detector 1 measures the time and spatial (x, y) location ofone part of an entangled pair that has interacted with the remote scene,target or subject. Box 4910 indicates spatially resolving detector 2.Spatially resolving detector 2 measures time and spatial (x, y) locationof the second part of an entangled pair that has interacted with theremote scene, target or subject. Box 4911 indicates coincidence andtiming electronics that operates to register when a pixel on detector 1and a pixel on detector 2 occur inside within a user defined coincidencewindow ΔT_(c). A coincidence window is a time difference within whichtwo photon measurements are defined to be co-incident. The timingelectronics further operate to record the time that has elapsed since auser chosen laser pulse and the first coincidence pair detection forranging calculations. Box 4912 indicates a processor, memory, andalgorithms to generate enhanced average second-order images of theregion of interest. Box 4913 indicates memory associated with processor4912 to store input images, algorithms, intermediate computations, andenhanced second order images of the region of interest. Box 4914indicates software operationally configured to perform the imageimprovement and enhancement processes. Box 4915 is a displayoperationally connected to processor 4912 to display the generatedenhanced second-order or higher order image of the region of interest.

FIG. 51 presents a block diagram of an embodiment of a system for imageimprovement. Box 4901 indicates a distant target scene area. Box 4902indicates the target which is an area of the distant target scene forthe particular region of interest selected for improvement andenhancement. Box 4903 indicates optionally present turbulence or otherobscuring or image degrading effects along photon path between lightsource and target, and light source and imaging system or photonmeasurement devices or components. 4904A is a telescope (although otherapparatus or direct viewing may be used without departing from the scopeof the invention). Telescope 4904A may zoom in on or expand view of theselected region of interest, as well as zoom out. It is used to transmitilluminating photons to the region of interest. Box 4904B is a secondtelescope (although other apparatus or direct viewing may be usedwithout departing from the scope of the invention). Telescope 4904B mayzoom in on or expand view of the selected region of interest. 4904B isconfigured to receive illuminating photons reflected or scattered fromthe region of interest. Box 4906A Illuminating photon source: Theilluminating photon source generates photons with distinguishableproperties such as two or more wavelengths, polarizations orentanglements of conjugate photon properties. Illuminating light may befrom one or more light sources either natural or artificial, or both.Common sources of light include, for example, sunlight, coherent,incoherent, or partially coherent light, entangled photons, infraredradiation emitted by atoms and molecules, accelerating charges, lasers,light bulbs, light emitting diodes (LEDs), chaotic laser light,pseudo-thermal light generated by passing laser light through a rotatingground glass or other scattering material, stars, moons, clouds,planets, space objects, fluorescent lamps, electrical discharges,plasmas, bio-luminescence, and stimulated emission. The use of anilluminating source other than an entangled photon source could provideadvantages when, for example, (a) a lower cost configuration isdesirable or (b) for cases where there is no entangled photon sourcewith the desired wavelength.

Box 4907 is a polarizing beamsplitter, dichroic-mirror or other opticalelement that operates to direct one portion of the photons withdistinguishable properties towards spatially resolving detector 1 anddirects the remaining portion of the photons with distinguishableproperties toward spatially resolving detector 2. Element 4908 is a lensused to focus the photons onto detector 1 and detector 2. Box 4909indicates spatially resolving detector 1. Spatially resolving detector 1measures the time and spatial (x, y) location of one part of anentangled pair that has interacted with the remote scene, target orsubject. Box 4910 indicates spatially resolving detector 2. Spatiallyresolving detector 2 measures time and spatial (x, y) location of thesecond part of an entangled pair that has interacted with the remotescene, target or subject. Box 4911A indicates image measurement,readout, and timing electronics that operates to read the measurementsfrom detector 1 and detector 2. Box 4911A differs from Box 4911 in thatBox 4911 includes a capability to register coincident measurementsbetween pixels located on detectors 4909 and 4910. Coincidenceelectronics are typically more complex and expensive than usual timingelectronics and are often only employed when a very high degree ofprecision in determining if two measurement events happened within acoincidence window ΔT_(c) is required.

The images from detector 1 and detector 2 are transferred to processor4912. Box 4912 indicates a processor, memory, and algorithms to generateenhanced average second-order images of the region of interest. Box 4913indicates memory associated with processor 4912 to store input images,algorithms, intermediate computations, and enhanced second order imagesof the region of interest. Box 4914 indicates software operationallyconfigured to perform the image improvement and enhancement processes.Box 4915 is a display operationally connected to processor 4912 todisplay the generated enhanced second-order or higher order image of theregion of interest.

FIG. 52 presents a flow chart block diagram describing a process toiteratively compute an improved image of the region of interest.Iterative methods can help generate a clearer image of a subject usingfewer frames of measurements than would otherwise be needed and canfurther produce a higher contrast improved image of the region ofinterest.

Box 5201 accepts an initial G⁽²⁾ image provided by, for example theprocesses described in FIGS. 40-41, and/or FIGS. 45-46. FIGS. 40-41 andFIGS. 45-46 are exemplary means to compute an initial G⁽²⁾ image, inparticular FIGS. 47 to 51 and 54 may use the methods described by FIGS.40-41 and FIGS. 45-46 or other methods to compute the initial G⁽²⁾image. In Box 5202 pixel locations within the provided G⁽²⁾ image areselected based upon a predetermined criteria such as those pixellocations where the value of the G⁽²⁾ image is greater than 0. At leastone pixel would need to be selected in Box 5202 to begin the iterativeprocess. Should no pixels be selected this can be construed to mean thatthe G⁽²⁾ image is already converged to an image where further iterationwould provide no corresponding improvement or enhancement to thegenerated image. In 5203 the measured image intensities (first arrayvalues) at the selected pixel locations are summed to provide new“bucket” values, “over-all frame intensity values” or “normalized pixelintensity sum” values. In 5204 a new G⁽²⁾ image is calculated usingcurrent computed “bucket” values. Box 5205 performs a determination aswhether to complete the iteration process which may include, forexample, testing all values of the new G⁽²⁾ image being positive, or thepixels selected for the current “bucket” values have not changed fromthe previous pixel location selected, or the calculated using currentcomputed “bucket” values or that the currently computed G⁽²⁾ image hasnot changed significantly from the previous computed G⁽²⁾ image, i.e.the maximum difference between the two images is less than 1e-6. If thetest is true then the process proceeds to box 5206, otherwise if thetest is false then the processes continues to box 5202. In Box 5206 theiteration process is complete and the computed G⁽²⁾ image of the regionof interest is available for storage or display.

FIG. 53 presents a flow-chart block diagram describing a process to usesub-ensembles to generate an improved image of the region of interest. Asub-ensemble process could be employed, for instance, to generate animproved image of a target that is a nearly instantaneous improved imageof the target. In other instances, sub-ensemble may be used to generatean improved image of a moving subject as a “freeze frame” withsignificant background subtraction. Also, the sub-ensembled process canfurther provide an indication of where a moving target is going and/orwhere the moving target has been.

In Box 5301, a sequence of images taken of the region of interest areprovided. Box 5302 is the start of the loop to compute per-frame newG⁽²⁾ images for a set of M frames where M<N (the total number of framesin the input sequence of images). In Box 5303 calculate the per-frameG⁽²⁾ image where I₁ is the measured pixel intensity of the kth frame. I₂is the overall frame intensity or “bucket” value of the kth frame. <I₁>and <I₂> are the overall averages of the measured pixel intensities andthe overall average of the “bucket” values where the average is takenover all N frames in the input sequence of images. In Box 5304 theper-frame sub-ensemble G⁽²⁾ images are summed over the M computed newG⁽²⁾ images. Box 5305 divides the sum of the per-frame sub-ensemble newG⁽²⁾ images by the number of frames used in the sub-ensemble (M) togenerate a sub-ensemble improved image of the region of interest. Box5306 ends the process and the computed sub-ensemble G⁽²⁾ image of theregion of interest is available for storage or display.

FIG. 54 presents a block diagram of an embodiment of a system for imageimprovement. This system is somewhat analogous to those shown in FIG.19A and FIG. 19B. The primary difference here, though, is the inclusionof active illumination components 4906A and 4906B. Examples of when asystem of this configuration could be used would include but is notlimited to cases when (a) the environmental, e.g. solar, lunar, etc,illumination is inadequate, (b) when a 3D time-of-flight ranged depthimage of the distant target area is being generated, (c) when an imageof particular colors of the distant target area is being generated, i.e.a wavelength dependent reflectance map, (d) when a particularfluorescence image of the distant target area is being generated,fluorescence of molecules/atoms is typically very specific to thewavelength of the illuminating light, (e) when a simpler possibly lessexpensive configuration is desirable, or (f) when a more compactconfiguration is required.

Box 4901 indicates a distant target scene area. Box 4902 indicates thetarget which is an area of the distant target scene for the particularregion of interest selected for improvement and enhancement. Box 4903indicates optionally present turbulence or other obscuring or imagedegrading effects along photon path between light source and target, andlight source and imaging system or photon measurement devices orcomponents. 4904A is a first telescope (although other apparatus ordirect viewing may be used without departing from the scope of theinvention). Telescope 4904A may zoom in on or expand view of theselected region of interest, as well as zoom out. Used to transmitilluminating photons to the region of interest. Box 4904B is a secondtelescope (although other apparatus or direct viewing may be usedwithout departing from the scope of the invention). Telescope 4904B mayzoom in on or expand view of the selected region of interest. 4904B isconfigured to receive illuminating photons reflected or scattered fromthe region of interest. Box 4906A Illuminating photon source. Theilluminating photon source generates photons with distinguishableproperties such as two or more wavelengths, polarizations orentanglements of conjugate photon properties. Element 4908 is a lensused to focus the photons onto detector 1. Box 4909A indicates spatiallyresolving detector 1. Spatially resolving detector 1 measures the timeand spatial (x, y) location of one photons that have interacted with theremote scene, target or subject. Detector 1A may be able to resolvewavelengths (color) or polarizations, etc. Box 4911B indicates imagemeasurement, readout, and timing electronics that operates to read themeasurements from detector 1. The timing electronics further operate torecord the time since a user chosen illumination pulse and the firstphoton detections for ranging calculations. The images from detector 1are transferred to processor 4912. Box 4912 indicates a processor,memory, and algorithms to generate enhanced average second-order imagesof the region of interest. Box 4913 indicates memory associated withprocessor 4912 to store input images, algorithms, intermediatecomputations, and enhanced second order images of the region ofinterest. Box 4914 indicates software instructions for operationallyconfiguring a processor to perform the image improvement and enhancementprocesses. Box 4915 is a display operationally connected to processor4912 to display the generated enhanced second-order or higher orderimage of the region of interest.

G⁽²⁾ type images may be calculated on a per-pixel basis by shifting thepixels in a frame of data, for instance if a frame of measured is I₁(i,j, k) where i, j indicates a pixel location and k is the time or frameindex, then I₂ can be set to be equal to a pixel shifted set of measuredvalues I₂(i, j, k)=I₁(i+1, j, k). Deviations of I₁ and I₂ from theirrespective ensemble averages can then be computed and the average of theproducts of those deviations <ΔI₁(i, j, k) ΔI₂(i, j, k)> is a G⁽²⁾ imagecomputed from space or coordinate shifted pixels of the measured values.Other statistical quantities such as standard deviations may also becomputed from the spatially shifted arrays.

The concept of time/frame shift of pixel values enhancement can bebeneficial in identifying objects in a scene that are in motion.Shifting of pixel values can also occur between frames wherein, forinstance, I₂(i, j, k)=I₁(i, j, k+1). Deviations and averages of theproducts of those deviations can be computed to yield G⁽²⁾ images of thearea of interest.

Normalizations

G⁽²⁾ improved images can be normalized by dividing the improved G⁽²⁾image by, for example, the product of the standard deviations of thereference values and the bucket values or dividing by the product of theensemble averages of the reference values and the bucket values. Othernormalizations, i.e., G⁽²⁾/(σ(I₁)* σ (I₂)) where I₁ and I₂ can be pixelvalues over time and G⁽²⁾/[(σ(I₁)*σ(I₂) *<I₁>*<I₂>)] Where < > indicatesan ensemble average and σ indicates the standard deviation. σ istypically computing using the following equation:

${\sigma (I)} = {\left( {\frac{1}{N}{\sum\limits_{l = 1}^{N}\left( {I - {\langle I\rangle}} \right)^{2}}} \right)^{\frac{1}{2}}.}$

G⁽²⁾ images can also be normalized by dividing by an all-positive G⁽²⁾to enhance image features. All positive G⁽²⁾ images can also benormalized by dividing by (σ(I₁) *σ(I₂)*<I₁>*<I₂>); this type ofnormalization on the all positive G⁽²⁾ calculations is particularlylifelike. It is noted that G⁽²⁾ images that are normalized by theproduct of the averages typically need to be colormap inverted, i.e.multiplied by −1, to recover realistic shading. G⁽²⁾ images that arecomputed with a pixel shift process or the bucket process can be used tonormalize the other calculation to highlight differences in the spatialor temporal scales of any distorting media between the measurementcomponent and the area of interest.

Subensembles

Sub-ensembles (such as utilizing a portion of an full ensemble, e.g.,1000 frames) can be averaged subset blocks ofG⁽²⁾/(σ(I₁)*σ(I₂)<I₁>*<I₂>), i.e., 10 blocks of 10 frames for frames 1to 100. Examples of sub-ensemble G⁽²⁾ s are as follows.

An example of a sub-ensemble used in a preferred method of the presentinvention comprises utilizing sum of G⁽²⁾ per frame where G⁽²⁾(k)=I₁(k)*I₂(k)−<I₁><I₂> and the sum of the G⁽²⁾ (k) over a set of i=1to total number of frames in the ensemble is the improved image. Asmaller subset of G⁽²⁾ (k) can be summed over a sub-ensemble, k=1 to Nwhere N is less than the total number of frames in the ensemble toprovide an improved image. Note that the full set of measurements may begreater than the N being used for the current ensemble.

A further example of a sub-ensemble used in a preferred method of thepresent invention comprises a G⁽²⁾ image that may be computed by summingand averaging per-frame G⁽²⁾ images over a set of frames that is lessthan the total number of frames in the ensemble of measurements. A perframe G⁽²⁾ would be G⁽²⁾ (k)=I₁(k)*I₂(k)−<I₁><I₂> where the k indicatesthe frame number out of the ensemble of frames. <I₁> and <I₂> are the“reference” and “bucket” averages over the ensemble of measurements.When the G⁽²⁾ (k) are summed and averaged over a set of frames that isless than the total number of frame this constitutes a sub-ensemble G⁽²⁾calculation. This sub-ensemble improved G⁽²⁾ image can highlightdifferent features of the area of interest and may further provide ameans to generate improved image frames for a movie of the area ofinterest.

The movies produced by this method can separate scale of turbulence.Separate scales of turbulence can be better images by the ability tochoose different sizes of an ensemble and different size sets within anensemble to use as the sub-ensemble.

Iterative Improvement Techniques

An iterative means to calculate in improved G⁽²⁾ image could involveprogressively incorporating more or fewer pixels of the measured framesto be summed over for a bucket value. One such method would be to choosea single pixel location to provide the bucket values over all frames. AG⁽²⁾ image would then be computed using those bucket values. A new setof bucket values would be selected based on some criteria, such aschoosing pixel locations where the G⁽²⁾ value is greater than 0. A newG⁽²⁾ image is computed and the process iterated until no new pixels arebeing added to or removed from the set of pixels being used to providebucket values. Criteria other than the positivity of the G⁽²⁾ may beused to emphasize or converge to differing features in the G⁽²⁾ image ofthe area of interest.

Iterative techniques may generally refer to methods wherein a sequenceof operations is repeated to achieve a desired result. These types ofmethods sometimes have stopping criteria that are based on an estimatederror parameter falling below some specified value. When this criterionis met the iteration is said to have “converged” to a solution. However,due to sometimes slow or erratic convergence a counter of the number oftimes the sequence of operations has been performed is often tracked.This number is sometimes called the iteration number. A further haltingcriterion beyond that of the iterative technique achieving the specifiedconvergence is a maximum iteration number. This maximum iteration numbercriterion is desirable when computational operations must be completedwithin a specified time and to prevent so called “infinite loops” wherethe iterative technique is unable, due to round off or other error, toachieve the convergence criterion.

Area of Interest as an Imperfect Mirror

When an area of interest is illuminated by a source of photons such asfor example, the Sun, laser, LED, etc, the illuminated area acts as animperfect mirror of the illuminating source. A perfect mirror would actsto reflect the photons from the illuminating source where the angle ofincidence would equal the angle of reflection. Furthermore, the perfectmirror would not absorb any of the incident photons. However, a regionof interest may contain many differing scattering and absorbingelements. All of these elements are illuminated by the source and whenthe reflected and scattered photons from the region of interest aremeasured some information about the spatial distribution of photons ofthe light source is retained. It is this retained information of thespatial distribution of photons that enables a series of images measuredwith a single sensor of a region of interest to act much as a typicaltwo sensor ghost imaging system wherein one of the sensors for a ghostimager measures only undisturbed spatial distributions of photons fromthe light source.

FIG. 55 shows an exemplary 3rd order enhanced imaging method embodiment.In step 5501, a series of color frames (e.g., 1-100) of a given scene,such as for example, photography that was influenced by the effect ofturbulence or the like, is input. Next, in step 5502, the color framedata may be partitioned into arrays of red (R), green (G) and blue (B)pixel values according to the color scheme. The product of the R, G andB pixel intensity values, R*G*B, is computed on each frame for allframes in step 5503. In step 5504, the R*G*B pixel values are summedover all frames on data on a per pixel based. In step 5505, the sum ofthe per pixel R*G*B pixel products is divided by the number of frames toyield and average <RGB> array.

Next, in step 5506, each R, G, B value per pixel is summed over allframes and stored in an array, RS, GS, BS. The array RS, GS and BS aredivided by the number of frames to yield average <R> <G> and <B> arraysin step 5507. Finally, in step 5508, the per pixel product of<R>*<G>*<B> is subtracted from the average <RGB> array to yield a thirdorder image of the scene.

FIG. 56 shows another exemplary 3rd order enhanced imaging methodembodiment. This is an exemplary embodiment for cases wherein monochromeor single wavelength band images are used. The second (I2) and third(I3) measurement values are taken, in this example, to be sums ofselected pixel locations. In step 5601, a series of frames (e.g., 1-100)of a given scene, such as for example, photography that was influencedby the effect of turbulence or the like, is input. In step 5602, theframe data is partitioned into 3 arrays, I1, I2 and I3. I1 is an arrayhaving pixel values for each frame. I2 is the sum of the values for aselected set of pixel locations. I3 is the sum of a second set ofselected pixel locations.

In step 5603, the product of the I1, I2 and I3 array values are computedon each frame for all frames. Next, in step 5604, the I1*I2*I3 arrayvalues are summed over all frames on data on a per pixel basis. In step5605, the sum of the I1*I2*I3 pixel products is divided by the number offrames to yield an average <I1*I2*I3> array. In step 5606, each I1, I2I3 value per pixel is summed over all frames and stored in an array,I1S, I2S, I3S.

In step 5607, the array I1S and the sums I2S, I3S are divided by thenumber of frames to yield average <I1> array and <I2>, <I3> values.Last, in step 5608, the per pixel product of <I1>*<I2>*<I3> issubtracted from the average <I1*I2*I3> array to yield a third orderimage of the scene.

FIG. 57 shows an exemplary 4^(th) order enhanced imaging methodembodiment. Fourth order enhanced imaging can allow for higher contrastimproved images to be generated using fewer measured frames of data than2^(nd) or 3^(rd) order enhanced imaging. In step 5701, a series of colorframes (e.g., 1-100) of a given scene, such as for example, photographythat was influenced by the effect of turbulence or the like, is input.

In step 5702, the color frame data is partitioned into 2 arrays, e.g.,red (R) and green (G) pixel values. In step 5703, the R and G pixelvalue arrays are summed for each frame to yield bucket value arrays, RBand GB. In step 5704, the product of the R*RB*G*GB values are computedon each frame for all frames.

Continuing to step 5705, the R*RB*G*GB pixel values are summed over allframes on data on a per pixel basis. In step 5706, the sum of the perpixel R*RB*G*GB pixel products are divided by the number of frames toyield an average <R*RB*G*GB> array. In step 5707, each RB and GB valuesis summed for all frames and stored as variables RBS, GBS. In step 5708,each R, G value per pixel is summed over all frames and stored in anarray, RS, GS.

Next, in step 5709, the arrays RS and GS are divided by the number offrames to yield average <RB> AND <GB>; and the variables RBS AND GBS aredivided by the number of frames to yield average <RBS> <GBS>.

Lastly, in step 5710, the per pixel product of <R>*<RBS*<G>*<GBS> issubtracted from the average array <R*RB*G*GB> to yield a fourth orderimage of the scene.

FIGS. 58-63 show various images for comparison sake in accordance withembodiments of the present invention. FIG. 58 shows an averaged image ofa target area with data acquired in turbulent conditions viewed from2.33 km away.

FIG. 59 shows an image of the same target area produced using thirdorder imaging of the method illustrated in FIG. 55, where(ΔI₁*ΔI₂*ΔI₃)/[σ(I₁)*σ(I₂)*σ(I₃)]. FIG. 60 shows an image of the sametarget area produce using third order imaging of the method illustratedin FIG. 56 where<I*bucket(I(70,35))*bucket(I(30,312))>−<I><bucket(I(70,35))><bucket(I(30,312)>. FIGS. 61 and 62 are images produced using the bucketdata used to produce the image in FIG. 60, in which FIG. 61 was producedwhere<I*bucket(I(70,35))*bucket(I(30,312))>−<I><bucket(I(70,35))><bucket(I(30,312)>and FIG. 62 was produced where <I*bucket(I(70,35))>−<I><bucket(I(70,35))>. FIG. 63 shows a shifted G⁽²⁾ image produced whereΔI(+1,:)*ΔI*ΔI(:,−1).

FIGS. 64-67 show various images for comparison sake in accordance withembodiments of the present invention. FIG. 64 shows an averaged image ofsink drain with images measured through flowing water. FIG. 65 shows asample single image of sink drain measured through flowing water. FIG.66 shows a sample G⁽²⁾ image of sink drain with images measured throughflowing water. FIG. 67 shows a sample G⁽²⁾ image normalized by productsof standard deviations computed using pixel shifts of sink drain withimages measured through flowing water.

FIG. 68 shows another exemplary 3^(rd) order enhanced imaging methodembodiment result. The image displays the R+++ conditional product term.This is an exemplary embodiment for cases wherein monochrome or singlewavelength band images are used. The second (I₂) and third (I₃)measurement values are taken, in this example, to be sums of selectedpixel locations.

FIG. 69 shows an exemplary 2^(nd) order image using 2 frames of data.The data in each frame was shifted by 10 pixels relative to each otherand a 2^(nd) order enhanced imaging method was used to generate thedisplayed image. The displayed image indicates a degree of backgroundsuppression and emphasis of some features of the target.

FIG. 70 shows an image of the target that was shifted by ten pixels todemonstrate that apparent motion of a target helps enable emphasis ofedges.

FIG. 71 shows an exemplary all positive 2^(nd) order image using 2frames of data. The background and nearly constant intensity features inthis figure are nearly fully suppressed and colored black. The brightlines are the edges of the target that have moved relative to each otherin the two input frames of data. Note that the edges are emphasized to agreater degree than in FIG. 69.

First Entangled Photons

Entangled photons may be used to determine range or distance to a remotetarget. In one embodiment, a pair of entangled photons with nearly thesame transverse momentum from an entangled photon source can becollinearly-propagated towards a remote target. The entangled photonsmay be directed towards specific areas on the remote target using apointing and aiming device. The entangle photons that reflect off of aremote target can be measured by photon detectors. When the entangledphoton source operates in a pulsed manner then an elapsed time from aparticular pulse to the first entangled photon pair measurement thatreturns from target may be determined. The time that it takes from thepulsed source to the target and back to the photon detectors indicatesthe distance of the target from the entangled photon source and detectorsystem. Since entangled photons may be recognized by consideration thatthe entangled photons are formed at virtually the same time andtherefore if they take the same path they must arrive at the detectorsat nearly the same time or in coincidence. An advantage of this type ofcoincidence measurement is that background noise from single photonmeasurements can be ignored and only detection measurements that happenin pairs within some small time interval will be used to provide a moreaccurate determination of distance to a remote target. Consider, forinstance, the case where a single pulse of entangle photons ispropagated and when the pulse contains more than one pair of entangledphotons then several coincidences may be measured after reflecting offof the remote target. The first pair of entangled photons to be detectedwould be indicative of reflection of the entangled photons from a nearertarget distance than later entangled photon measurements generated bythe same pulse. The locus of target points indicated by the detectionevents would describe the shape and distance of the target from thesetup comprising the entangled photon source and detectors. Photons thatwere received after the first entangled photons are indicative ofphotons that have traveled further than the nearest points of the targetand may have been scattered by the environment or within the body of thetarget over a longer path before being measured. Scatterers in theenvironment may include the atmosphere, dust, fog, air pollution, etc.It should be appreciated that this process would apply to propagation inwater, liquids, oceans, solids such as glass, and other media such asbiological tissues.

In the non-collinearly propagating entangled photon case when thephotons are directed towards a remote target then each photon of thepair likely will be reflected off of different areas of the remotetarget.

Two detectors can measure the photons in a small time interval, i.e.coincidence time window. One detector can act as a bucket detector andthe other detector can determine the reference location of the measuredphoton. Thus the entangled photons can be used for ghost imaging,higher-order imaging, but also determine the distance of each measuredpixel from the entangled photon source to the detectors setup. Ofcourse, the timing must consider the path from the source to the targetand back to the detectors.

The first entangled photon pair back may be calculated from the time ofthe pulse. One or more optical delays may be included to allow a longerpath and thereby allow longer time delays to measure a coincidenceevent. Typical optical delays that may be used are for example, lenses,quartz rods, fiber-free space optical delay line, slow light media, orquantum memories.

Motion of Target in a G⁽²⁾ Image

Ghost imaging, sometimes called higher order or G⁽²⁾ imaging, may beused to identify the motion of objects within a region of interest. Ifthe time period for forming a ghost imaging is too short then the motioncannot be detected in a single ghost imaging frame. A ghost imagingmovie can be made by stringing together such a series of frames coveringa time period over which an object is moving. This way, the objectmotion does not smear the ghost image but rather resolves each of theindividual frames in the series. However, there is another way toindicate motion of an object within a region of interest via ghostimaging. If the object moves at least one pixel, a series of measurementframes are collected, then a ghost image using those frames may resolvethe object. But signatures of the object motion will appear on the ghostimage. In the case of the all-positive form for a ghost image usingconditional products, (R_(pp)+R_(nn)−(R_(np)+R_(pn))), then edges of themoving object can be distinguished by above background positivemagnitudes. The ghost image made from positive and negative quadrants,(R_(pp)+R_(nn)+(R_(np)+R_(pn))), will still visually appear as animproved image. The separate terms, R_(pp), R_(nn), R_(np), R_(pn), mayhave both edge and improved image qualities. It is to be noted that G⁽²⁾may take on both positive and negative values depending on locationwithin the image. That is, some points will be positive or zero valuedand other points will have negative values. The ghost images producedcan still acquire the turbulence free ghost imaging properties eventhough the signs may change. That is the images will still be enhancedand improved over conventional images. A constant can be added to theghost image to guarantee its positivity to satisfy certain renderingrequirements. Thus, ghost imaging of moving objects has the benefit inthat it can produce edge maps of the scene and indicate objects that aremoving such as to separate them from stationary objects in the sameregion of interest. Such edge maps and identification of moving objectscan be very useful for many human endeavors requiring such benefits thatmay include machine vision for robotics,intelligence-surveillance-reconnaissance, moving object pointing andtracking, medical imaging of moving organs, law enforcement, militarysituational awareness, documentation of flow lines of moving media,earthquake motion, or missile motion. For example, transmission of asingle ghost image identifying moving objects in a scene is moreefficient than transmitting a movie with many frames. As a furtherexample the edge image generated by ghost imaging can be mapped to a 1bit per pixel image where only those edges greater than some thresholdare set to a value of 1 and all other pixels are set to zero to greatlyincrease transmission efficiency. Ghost imaging of the Earth fromsatellite or aircraft can indicate changes in terrain features and cropevolution. Ghost images of outer space from the Earth may indicate theoutlines of moving satellites more clearly and distinguish them frombackground. Ghost imaging can also be applied to indicate the motion ofunderwater objects more clearly, or objects in moving media moreclearly.

It must be appreciated that all-positive enhanced images for higherorder, i.e. G⁽³⁾, G⁽⁴⁾, . . . ,G^((N)), images can be calculated bycombining the products of the higher order quadrant terms may be groupedsuch that an individual conditional product term (R_(m)) has a positiveor a negative value. The all positive G^((N)) can then be determined byadding together those R_(m) terms with positive values and subtractingthe negative valued R_(m) terms. In the following R_(m) is shorted to Rfor ease is reading. For example an all positive G⁽³⁾ would have R⁺⁺⁺,R⁻⁻⁻, R⁺⁺⁻, R⁺⁻⁺, R⁻⁺⁺, R⁻⁻⁺, R⁻⁺⁻, R⁺⁻⁻, conditional product terms. Theconditional product terms that have positive values are R⁺⁺⁺, R⁻⁻⁺, R⁻⁺⁻and, R⁺⁻⁻. The conditional terms with negative values are R⁻⁻⁻, R⁺⁺⁻,R⁺⁻⁺, and R⁻⁺⁺. So an all-positive G⁽³⁾ would have the form(R⁺⁺⁺+R⁻⁻⁺+R⁻⁺⁻+R⁺⁻⁻)−(R⁻⁻⁻+R⁺⁺⁻+R⁺⁻⁺+R⁻⁺⁺). Similar groupings can bedetermined and used to calculate other all-positive G(N) enhancedimages.

FIG. 72 presents a block diagram of an embodiment of a system for imageand ranging improvement. Box 4901 indicates a distant target scene area.Box 4902 indicates the target which is an area of the distant targetscene for the particular region of interest selected for improvement andenhancement. Box 4903 indicates optionally present turbulence or otherobscuring or image degrading effects along photon path between lightsource and target, and light source and imaging system or photonmeasurement devices or components. Box 4904A is a first telescope(although other apparatus or direct viewing may be used withoutdeparting from the scope of the invention). Telescope 4904A may zoom inon or expand view of the selected region of interest, as well as zoomout. Used to transmit entangled photon pairs to the region of interest.Box 4904B is a second telescope (although other apparatus or directviewing may be used without departing from the scope of the invention).Telescope 4904B may zoom in on or expand view of the selected region ofinterest. 4904B is configured to receive entangled photon pairsreflected or scattered from the region of interest. By providingseparate first and second telescopes, it is possible to provide betterresults and control independently for light transmitted and received.

Box 4906 is an entangled photon source. The entangled photon sourcegenerates entangled photon pair that are entangled in time-energy, H-Vpolarization or between other conjugate pair properties of the photons.Box 4907 is a polarizing beamsplitter, dichroic-mirror or other opticalelement that operates to direct one portion of an entangled photon pairtowards spatially resolving detector 1 and directs the remaining portionof an entangled photon pair toward spatially resolving detector 2.Element 4908 is a lens used to focus the photons onto detector 1 anddetector 2. Box 4909 indicates spatially resolving detector 1. Spatiallyresolving detector 1 measures the time and spatial (x, y) location ofone part of an entangled pair that has interacted with the remote scene,target or subject. Box 4910 indicates spatially resolving detector 2.Spatially resolving detector 2 measures time and spatial (x, y) locationof the second part of an entangled pair that has interacted with theremote scene, target or subject. Box 4911C indicates coincidence andtiming electronics that operates to register when a pixel on detector 1and a pixel on detector 2 occur inside within a user defined coincidencewindow ΔT_(c). A coincidence window is a time difference within whichtwo photon measurements are defined to be co-incident. The timingelectronics further operate to record the time that has elapsed since auser chosen laser pulse and the first coincidence pair detection forranging calculations. Box 4912 indicates a processor, memory, andalgorithms to generate enhanced average second-order images of theregion of interest. Box 4913 indicates memory associated with processor4912 to store input images, algorithms, intermediate computations, andenhanced second order images of the region of interest. Box 4914indicates software operationally configured to perform the imageimprovement and enhancement processes. Box 4915 is a displayoperationally connected to processor 4912 to display the generatedenhanced second-order or higher order image of the region of interest.Box 4916 indicates optionally-present pointing and aiming (e.g., beamsteering) components that may be used to direct the entangled photonpairs to a specific point within the region of interest.

Filtering may be used in various embodiments for enhanced imageprocessing. The filtering processes can be performed electronically,digitally, computationally, or a combination thereof, for example, andmay be performed at different stages of processing. In some embodiments,quantum filtering techniques may be employed, including Quantumcomputations on classic computers as discussed, for instance, in U.S.Pat. No. 7,353,148.

As an example, consider that detector 1 and detector 2 where eachdetector can measure photons. The measurements from the two detectorsmay be used to determine G⁽²⁾ correlations of the measured photons. G⁽²⁾correlations appear when the photons are measured within a specifiedtime interval of each other. The G⁽²⁾ correlations can be used togenerate enhanced images, as discussed above. As shown in FIGS. 73-77,filtering can further improve the enhanced images.

FIG. 73 shows a comparison of filtered “bucket” values vs. time (dottedline) and unfiltered “bucket” values vs. time (solid line). The “bucket”values represent intensity values, and generally will be propotional tophoton counts. As will be appreciated, the dotted line has fewerextremes and is generally smoother than the solid line, thusillustrating the benefit of filtering. The filtering may be performed,for example, according to the methodology of FIG. 78 discussed below.

FIG. 74 shows an example of results generated using seven frames with afiltering technique on the input measurements. It is noted thevisibility of the three vertical lines in (a), the letters “IOL” in (b)and in (c).

FIG. 75 shows an example of results generated using the same sevenframes of unfiltered input measurements. In comparison to FIGS. 74 (a),(b), and (c), it is noted that the three vertical lines and the “IOL”letters are unrecognizable.

FIG. 76 shows an example of the average image generated using afiltering technique on the input measurements.

FIG. 77 shows an example of the average image generated using aunfiltered input measurements.

FIG. 78 shows an exemplary embodiment of an exponential filteringprocess used on measured input values. At Box 7801 multiple frames ofmeasured pixel values are available from an input component. At Box 7802a loop over all input measurement locations I=1 to IMAX and J=1 to JMAXis started. In Box 7803 the first frame filtered pixel value is set tobe equal to the first frame pixel value of the measured pixel input. AtBox 7804 start a loop over the remaining K=2 to KMAX frames of measuredpixel values. In Box 7805 the Kth frame filtered pixel value (I, J, K)is set to be equal to exp(−dt/TF)*measured pixel value(I,J,K)+(1−exp(−dt/TF))*Filtered pixel value (I, J, K−1). At Box 7806,the method proceeds to next input pixel (I, J) measurements. And at Box7807, the filtered data is passed to the rest of calculations. At Box7808, the filter process is completed and the filtered measurementvalues are available to be used in place of or along with the unfilteredmeasurement values to generate and improved image of the region ofinterest. It should be appreciated that other forms of filtering such asChebyshev, Kalman, Fourier, etc., may be used in lieu of the exponentialfilter described here for illustrative purposes.

There may be situations, such as involving sunlight or incoherent light,in which the two detector measurements could be virtually uncorrelated.More particularly, the time-frequency properties of the detectors may besuch that they do not resolve the G⁽²⁾ correlations which would normallyyield an enhanced image. These detectors measurements in thesesituations may not produce a readily appreciable image. But theinventors have found that, with filtering, these measurements can stillnonetheless produce an enhanced image processing. Thus, in someembodiments filtering (i) the individual detector measurement eventsbefore determining the G⁽²⁾ correlations, and/or (ii) during thedetermination of the joint coincidences, may yield enhanced images sincethe resolvable filtered components of the G⁽²⁾ would be processed.

Entanglement Swapping

Entanglement swapping is a quantum process by which particles that arenot entangled become entangled with each other. For example, considerthat particles 1 (P1) and 2 (P2) are entangled with each other and thatparticles 3 (P3) and 4 (P4) are entangled with each other. To entangleP1 and P4, particles P2 and P3 are interfered on a beam splitter andthen are measured. The interference and measurement swaps theentanglements P1−P2 and P3−P4 to P1−P4. Particles P2 and P3 are alsoaffected by the measurement device and may be absorbed. The process ofentanglement swapping has previously been verified. See, e.g.,Bouwmeister et al. [Physical Review Letters 80, 3891-3894 May 1998]which described a process of entanglement swapping with experimentalverification using entangled photons. Swapping may be considered as theteleportation of an unknown photon/particle state onto anotherphoton/particle.

The process of entanglement swapping has many potential applications inthe development of quantum technology. Thus far, relatively fewapplications have found uses for entanglement swapping. Potentialapplications for entanglement swapping in quantum technology includequantum computing, quantum communications and, in the current invention,quantum imaging. There are potentially many benefits to usingentanglement swapping for quantum imaging that have not yet beendescribed or exploited. The reason for this is that entanglementswapping has required high precision in its implementation and greatexpense for equipment that achieves the high precision. The lack ofrobust applications for entanglement swapping has been another drawbackto its implementation in technology. This technology is beingminiaturized in solid state devices and some components are being testedon chips. These quantum chips, can generated entangled particles andperform interference operations and measurements of quantum states.

It would be beneficial to have an entanglement swapping application thatis robust and can be implemented with both available and evolvingtechnologies. One way to make entanglement swapping useful would be toapply it information transfer, sharing, or communication without theneed for a classical communications channel. For example, the currentInternet, radio, and telephone are generally considered to be classicalcommunications channels. Another way to make entanglement swappinguseful would be to be able to transfer, share or communicate by quantummeans without the sender or receiver needing access to information orresources held by the other. For example, the sender having access tophotons P2, P3 and the receiver having access to photons P1, P4 issufficient to transfer information from sender to receiver. Repetitionof this process allows the transfer of images without sending classicalinformation and by only sharing entanglement. This type of communicationof information, such as data and/or images, would be difficult to detectby an external observer since there would be no particle or radiationgoing between the sender and the receiver which an observer would beable to sense and follow. Military and domestic applications requiringstealth and/or security would benefit from this capability.

Benefits of entanglement swapping for quantum imaging may includeperforming an entanglement swap to optimize photon detection efficiencywhile simultaneously optimizing transmission properties from anillumination source to a target. Another benefit is that an entanglementswap may be used to measure absorption maps of a target without the needto measure reflected photons. Furthermore, entanglement swapping may beused to help compute the product of the absorption values at twolocations on a target. Using the environment to enable entanglementswapping provides a direct and remote measurement on the environment.For example, absorption of photons by a remote target can be sensed bythe enabling of quantum swapping of entangled particles which can bemeasured remotely without need for the return of photons from thetarget. It should be noted that besides images of absorption fields oftargets any property can be imaged by enabling quantum swapping when thequantum particle is sensitive to the effects of object. Furthermore,with time sequencing this provides range information from, for example,the source of entangled quantum particles to target features. It shouldbe further realized that the source or sources of the entangled quantumparticles need not be located with the equipment used to directparticles towards a target (sender) or located with the equipment thatmeasured those entangled particles that never directly interacted withthe target (receiver). For example, the source or sources of theentangled particles may be on a satellite that would send the entangledparticle pairs to the “sender” equipment and “receiver” equipment.Alternately, both the sender and receiver may have a single entangledquantum particle source and each shares one particle of their entangledparticle pairs with the other. The identification of which particles areentangled with each other relative to initial entangled pair creationtimes may be achieved using an auxiliary time stamp, e.g. a laser pulseencoded with time information for each entangled photon pair created,that propagates with each particle of each entangled particle pair.Also, the use of an entanglement source such as the one described inFIG. 81 does not have an issue (or question) as to the identification ofwhich particles are entangled as there is only a single source thatsequentially generates entangled particles. Although not obvious, weconsider it possible to use thermal light photon number fluctuations andtheir correlations and quantum illumination for variations ofteleportation and swapping in our current inventions with swapping.

Further benefits of entanglement swapping applied to quantum imagingusing measurements of reflected photons may include application toquantum imaging of remote targets and microscopy with the images beinggenerated for the user at a distant location with entangled photons thatdid not interact directly with the target. The reflected photons may befurther used to compute the product of reflectance or the product ofreflected intensities of at least two locations on the target. Currentimaging systems, such as cameras, are dependent on producing imagingusing photons that have directly interacted with the target. The sharingof images taken by a camera normally requires communication byelectromagnetic radiation that takes specific paths to communicate afacsimile of the image between sender and receiver. Even quantumteleportation requires a classical communication channel usingelectromagnetic radiation that takes specific paths to communicate. Itwould be beneficial to use entanglement swapping to communicate imagesor quantum images that does not require a classical communicationschannel to complete the transfer of images between a sender and adistant user at the receiver in order to avoid having the classicalcommunications channel blocked which would also block imagecommunication. Communication information transfer using entanglementswapping would be an entirely quantum process. The speed of quantuminformation has been recently been reported as being greater than orequal to 1.37*10⁴ times the speed of light See, J Yin et al. [PhysicalReview Letters 110, 260407 June 2013]. The benefits of utilizingswapping in the process of quantum communications is that communicationswould be at the speed of the quantum information even if it is fasterthan the speed of light which can be beneficial for many applications.

FIG. 79 presents a block diagram of an embodiment of a system for imageand ranging improvement. Box 4901 indicates a distant target scene area.Box 4902 indicates the target which is an area of the distant targetscene for the particular region of interest selected for improvement andenhancement. Box 4903 indicates optionally present turbulence or otherobscuring or image degrading effects along photon path between lightsource and target, and light source and imaging system or photonmeasurement devices or components. Box 4904A is a first telescope(although other apparatus or direct viewing may be used withoutdeparting from the scope of the invention). Telescope 4904A may zoom inon or expand view of the selected region of interest, as well as zoomout.

The system is specifically configured to transmit entangled photon pairsto the region of interest. Box 7906 is an entangled photon source, suchas of the types further illustrated by FIGS. 80 and 81. The entangledphoton source generates entangled photon pairs that are entangled intime-energy, H-V polarization or between other conjugate pair propertiesof the photons. Box 4907 is a polarizing beamsplitter, dichroic-mirroror other optical element that operates to direct one portion of anentangled photon pair towards spatially resolving detector 1 and directsthe remaining portion of an entangled photon pair toward spatiallyresolving detector 2. Boxes 4908 are lenses used to focus photons ontodetector 1 and detector 2. Box 4909 indicates spatially resolvingdetector 1. Spatially resolving detector 1 measures the time and spatial(x, y) location of one part of an entangled pair that has interactedwith the remote scene, target or subject. Box 4910 indicates spatiallyresolving detector 2. Spatially resolving detector 2 measures time andspatial (x, y) location of the second part of an entangled pair that hasinteracted with the remote scene, target or subject. Box 4911 indicatescoincidence and timing electronics that operates to register when apixel on detector 1 and a pixel on detector 2 occur inside within a userdefined coincidence window ΔT_(c). A coincidence window is a timedifference within which two photon measurements are defined to becoincident. The timing electronics further operate to record the timethat has elapsed since a user chosen laser pulse and the firstcoincidence pair detection for ranging calculations. Box 4912 indicatesa processor, memory, and algorithms to generate enhanced averagesecond-order images of the region of interest. Box 4913 indicates memoryassociated with processor 4912 to store input images, algorithms,intermediate computations, and enhanced second order images of theregion of interest. Box 4914 indicates software operationally configuredto perform the image improvement and enhancement processes. Box 4915 isa display operationally connected to processor 4912 to display thegenerated enhanced second-order or higher order image of the region ofinterest. Box 4916 indicates optionally-present pointing and aiming(e.g., beam steering) components that may be used to direct theentangled photon pairs to a specific point within the region ofinterest.

FIG. 80 shows an exemplary embodiment of an entangle photon source whichmay be used for Box 7906 FIG. 79. In particular, FIG. 80 presents anexpanded view of one embodiment for the generation of entanglementswapped photon pairs. Element 8001 includes two entagled photon sourcesEPS1 and EPS2. Boxes 8002 are optionally present phase modulatorsoperative to modify the phase relationship between the photon pairsgenerated by EPS1 and EPS2 respectively. Box 8003, for example, abeamsplitter, operates to interfere at least one photon of at least oneentangled photon pair from EPS1 with at least one photon of at least oneentangled photon pair from EPS2. Box 8004 is an optical delay line thatoperates to ensure photon overlap on beamsplitter 8003 for optimizinginterference. Box 8005 indicates that the photon pairs that haveinterfered on element 8003 are directed towards telescope 4904A and thento target A02. Box 8006 indicates that the remaining photons from EPS1and EPS2 are directed towards element 4907 and measurement devices 4909and 4910.

FIG. 81 shows an alternate exemplary embodiment for the generation ofentanglement swapped photon pairs which may be used for Box 7906 FIG.79. Element 8001 (EPS1) is a source of entangled photon pairs. It may beconfigured to “pulse” photons in time. Boxes 8002 are optionally presentphase modulators operative to modify the phase relationship betweenphoton pairs generated at different times by EPS1. Boxes 8100A and8100B, for example, a beamsplitter, operate to direct at least onephoton of the entangled photon pair towards Boxes 8003A and 8003B,respectively. Element 8003 (8003A or 8003B), for example, abeamsplitter, operates to interfere at least one photon of at least oneentangled photon pair from a first time from EPS1 with at least onephoton of at least one entangled photon pair from a second time. Box8004 is an optical delay line that operates to ensure photon overlap onelement 8003 for optimizing interference. Box 8005 indicates that thephotons that have interfered on element 8003 are directed towardstelescope 4904A and then to target A02. Box 8006 indicates that firsttime-second time entangled photons from EPS1 are directed towardselement 4907 and measurement devices 4909 and 4910. It is noted that theoptical path lengths from Boxes 8100A/B to 8003A/B should beapproximately an integer multiple of the time between entangled photonpulses generated by 8001 (EPS1), i.e. the time between entangled photonpair pulses is Δt_(p) then the path length is approximately 1, 2, 3, . .. , n times delta t_(p). Optical path lengths may be measured in termsof a distance L or time where the time is the length L divided by thespeed of light in the media c_(m).

FIG. 82 presents a block diagram of an alternate embodiment of a systemfor image and ranging improvement. Box 4901 indicates a distant targetscene area. Box 4902 indicates the target which is an area of thedistant target scene for the particular region of interest selected forimprovement and enhancement. Box 4903 indicates optionally presentturbulence or other obscuring or image degrading effects along photonpath between light source and target, and light source and imagingsystem or photon measurement devices or components. Box 4904A is a firsttelescope (although other apparatus or direct viewing may be usedwithout departing from the scope of the invention). Telescope 4904A mayzoom in on or expand view of the selected region of interest, as well aszoom out. Used to transmit entangled photon pairs to the region ofinterest. Box 4904B is a second telescope (although other apparatus ordirect viewing may be used without departing from the scope of theinvention). Telescope 4904B may zoom in on or expand view of theselected region of interest. The telescope 4904B is configured toreceive entangled photon pairs reflected or scattered from the region ofinterest. By providing separate first and second telescopes, it ispossible to provide better results and control independently for lighttransmitted and received.

Box 7906 is an entangled photon source, such as of the types illustratedby FIGS. 80 and 81. The entangled photon source generates entangledphoton pairs that are entangled in time-energy, H-V polarization orbetween other conjugate pair properties of the photons. Box 4907 is apolarizing beamsplitter, dichroic-mirror or other optical element thatoperates to direct one portion of an entangled photon pair towardsspatially resolving detector 1 and directs the remaining portion of anentangled photon pair toward spatially resolving detector 2. Element4908 is a lens used to focus the photons onto detector 1 and detector 2.Box 4909 indicates spatially resolving detector 1. Spatially resolvingdetector 1 measures the time and spatial (x, y) location of one part ofan entangled pair that has interacted with the remote scene, target orsubject. Box 4910 indicates spatially resolving detector 2. Spatiallyresolving detector 2 measures time and spatial (x, y) location of thesecond part of an entangled pair that has interacted with the remotescene, target or subject. Box 4911B indicates coincidence and timingelectronics that operates to register when a pixel on detector 1 and apixel on detector 2 occur inside within a user defined coincidencewindow ΔT_(c). A coincidence window is a time difference within whichtwo photon measurements are defined to be co-incident. The timingelectronics further operate to record the time that has elapsed since auser chosen laser pulse and the first coincidence pair detection forranging calculations. Box 4912 indicates a processor, memory, andalgorithms to generate enhanced average second-order images of theregion of interest. Box 4913 indicates memory associated with processor4912 to store input images, algorithms, intermediate computations, andenhanced second order images of the region of interest. Box 4914indicates software operationally configured to perform the imageimprovement and enhancement processes. Box 4915 is a displayoperationally connected to processor 4912 to display the generatedenhanced second-order or higher order image of the region of interest.Box 4916 indicates optionally-present pointing and aiming (e.g., beamsteering) components that may be used to direct the entangled photonpairs to a specific point within the region of interest.

A second set of entangled photon pairs generated by 7906 are directedtowards element 4907A. Element 4907A is similar to 4907 and directsportions of the entering entangled photon pairs towards measurementdevices 4909 and 4910. Coincidence measurements of the entangled photonsdirected to element 4907A are used to generate a reflection image of thetarget 4902 with information that is provided by the shared entanglementproperties of the entangled photons that were directed from 7906 totelescope 4904A. It is to be appreciated that this invention willoperate to generate an improved image of the target where the target maybe partially absorbing and/or partially reflecting.

Potential Applications

G⁽²⁾ imaging in the infrared may be useful for applications in: art,medical imaging of blood vessels, identify and face recognition, imagingof blood vessels, medical imaging of blood vessel flow over the face andother parts of the body, and remote feature extraction.

A further application is terahertz ghost imaging. The methods andtechniques of this invention apply to all electromagnetic wavelengths.Gamma rays, ultraviolet, visible, infrared, microwave, and radio wavesand terahertz radiation can be utilized to produce improved images of aregion of interest.

The potential extent of possible use of this invention is described inthe following. However, this description should not be construed aslimited to the statements. Potential applications include highresolution imaging, remote sensing, microscopic sensing, phase-contrastimaging or microscopy, astronomical imaging, physics, chemistry,biology, medical applications, quality control, surveillance, surfacetampering detection, imaging partially occluded scenes, spectroscopy,raman spectroscopy, satellite imaging, detection of exoplanets,identification of hidden or concealed objects, remote biometrics, designof new sensors and image processing methods, design of new types ofstealth technology, design of new types of communications devices.Furthermore, the methods and techniques can be used to determinecharacteristics of imaging sensors and discover favorable or unfavorableartifacts and properties including but not limited to spatial andtemporal noise.

Speed Traffic Enforcement—Current local governments use trafficenforcement cameras to enforce traffic regulation violations. A trafficenforcement camera (also road safety camera, road rule camera, photoradar, speed camera, Gatso™) is an automated ticketing machine. It mayinclude a camera which may be mounted besides, on, or over a highway orinstalled in an enforcement vehicle to detect traffic regulationviolations, including speeding, vehicles going through a red trafficlight, unauthorized use of a bus lane, for recording vehicles inside acongestion charge area and others. The latest automatic number platerecognition (ANPR) systems can be used for the detection of averagespeeds and use optical character recognition on images to read thelicense plates on vehicles. There are a number of possible factors thataffect the ANPR software performance. One of these important factors ispoor image resolution, usually because the plate is too far away butsometimes resulting from the use of a low-quality camera. In the case ofcamera recording a video (a sequence of images), this invention canprocess the recorded images to improve image quality of the licenseplate on vehicle. The enhanced license plate images are used to improvethe performance of ANPR software. The invention is especially usefulwhen the images are acquired from a far away distance and/or from alow-quality camera.

The invention may be utilized in conjunction with large crowd eventsecurity and management. Events involving a large crowd, especially thetypes of events including circuses, sporting events, theatrical events,concerts, rallies, parades, etc., the security task is to prevent, wherepossible, crimes including theft, vandalism or assault through thedeployment of trained and licensed security personnel. Camera monitoringis an important component in this type of event security and management.The invention can be used to improve image details of a human face,nomenclature on a jersey, or a moving object/vehicle, etc., from adistance, or from the periphery of the event location. Also at footballgames, a preferred embodiment could be used to enhance the readabilityof numbers and/or names on football uniforms.

As used herein, the terminology “subject” means: an area, a scene, anobject or objects, a landscape, overhead view of land or an object orobjects, or a combination thereof.

As used herein, the terminology “frame” means: a picture, an image orone of the successive pictures on a strip of film or video.

As used herein, the terminology “process” means an algorithm, software,subroutine, computer program, or methodology.

As used herein, the terminology “algorithm” means: sequence of stepsusing computer software, process, software, subroutine, computerprogram, or methodology.

As used herein, the terminology “image sensor” means: a camera, bucketdetector, CMOS, SPAD, quantum well, LIDAR, LADAR, charge coupled device(CCD), video device, spatial sensor, light field (plenoptic) camera,gyro-stabilized camera, spatial phase sensitive camera, or range sensor.The image sensor may comprise a device having a shutter controlledaperture that, when opened, admits light enabling an object to befocused, usually by means of a lens, onto a surface, thereby producing aphotographic image OR a device in which the picture is formed before itis changed into electric impulses.

The terminology “camera” as used in the following claims includesdevices which measure intensity of photons to produce images. Cameraswhich can take a series of images may be termed video cameras. Manyindividual cameras have circuitry to allow the capture and storage ofvideo sequences and hence are called video cameras. Cameras are oftenembedded or attached to computers or smart/cell phones or other types ofphones. Cameras may be sensitive to one or more wavelengths of light.Color cameras are sensitive to more than one wavelength band of lightand can distinguish between those wavelength bands. For example, colorcameras may sense red, green, and blue at separate pixels. Color camerasmay use a Bayer pattern for collection of more than one wavelength bandand use methods for Bayer interpolation to provide intensity for eachwavelength band at each pixel. Digital cameras are readily interfaced todigital computers. Analog cameras need analog-to-digital (A/D) circuitryto transfer images from the camera to a digital computer processor. Forexample, a digital camera is a device used for measuring images of aregion of interest. Some digital cameras such as charged coupled devices(CCDs) have an advantage of being able to achieve low image noiseespecially when cooled. High quality CCDs may have an electronic coolingcapability to lower image noise. CCD cameras may have readout orprocessing noise. CMOS cameras also called complementary metal oxidecameras. CMOS cameras may have an advantage of lower cost and may havelower readout noise but may have higher image noise. CMOS cameras canalso be cooled to achieve improved properties. Analog cameras have beenused in television industry and may have advantages where digitalcameras should not be used. For example, analog cameras may be lesssusceptible to digital computer viruses. Analog cameras may already beused as sensors in a wide variety of applications. Analog cameras may bemade using photo-diodes. Photo-diodes measure single or multiple photonsand in general the total intensity of light falling on the diode.Photo-diodes have a small or large active area which allows them to beused as point or area detectors such as photon bucket detectors.Plenoptic cameras utilize an alternate means to record information of ascene or region of interest than conventional cameras. Plenoptic camerasrecord light coming into the camera through more than one lens. Themeasurements are saved in a manner that can be combined to achieverefocusing of the image after the measurements are made. As plenopticcameras improve, they will be able to store measured light with higherresolution and speed.

The terminology “SPAD” as used in the following claims means SinglePhoton Avalanche Diode. SPAD arrays can be used to form an image of anarea of interest in terms of single photon counts for a specified periodof time at each SPAD location. Currently SPADs can operate in thenano-seconds and pico-seconds but it is expected that their speeds mayimprove even further. Since SPADs can be individually addressed withelectronic logic they may be suitable for a wide variety of multi-photoninterference applications such as for producing enhanced images in caseswith and without obscurants and/or turbulence. SPADs are becoming morecommon because the technology is advancing to lower the cost of SPADarrays. SPADs may also be coupled with illumination timing circuits suchas laser triggers to be able to measure time-of-flight for an emittedlaser pulse to travel to a target and return to a particular SPAD pixel.Thus each pixel of a SPAD array in this case can be used to represent adistance or depth map between the SPAD sensor and a correspondinglocation on a target in the region of interest. Such a distance or depthmap would indicate not only the intensity of light reflected andreturned but also the distance to the target at each correspondingpoint.

The terminology “LIDAR” as used in the following claims means LightDetection and Ranging. LIDAR devices use lasers to measure distances toan object. Often LIDARs are pulsed but they may alternatively bemodulated in periodic or aperiodic ways in order to determine distancesuch as by use of phase modulation. LIDARs may be scanned to produce animage and range map also giving depth and 3D information of the targetin the region of interest.

The terminology “LADAR” as used in the following claims means LaserRadar which produces an image or a map of objects at a distance from thelaser source. LADARs may use scanning or “flash” methods. Flash methodsare distinguished from scanning methods in that a larger region oftarget is illuminated at nearly the same time. Whereas a single laserbeam scan illuminates different parts of the target at sequential times.However, arrays of lasers may also be used to illuminate different partsof a target at approximately the same time. Each laser when used incombination with a synchronized sensor produces a distance to the targetin addition to a strength of intensity return.

The terminology “display device” as used in the following claims includea display capable of rendering a display. These ways may includerendering on a screen, cathode ray tube (CRT), glasses, liquid crystaldiodes (LCDs), light emitting diode (LED) arrays, plasma screens,projectors projecting onto a surface, or even to the eyes. Moderntelevisions (TVs), sometimes called smart TVs, not only render imagesfor viewing but also render the images into other formats which may beinterfaced to other imaging, multi-media, computing, or recordingdevices such as digital video recorders (DVRs), through free-space (e.g.WiFi or Bluetooth), electrical wiring, and optical fiber.

The terminology “processor” or “image processor” as used in thefollowing claims includes a computer, multiprocessor, CPU, GPU, FPGA,minicomputer, microprocessor or any machine similar to a computer orprocessor which is capable of processing algorithms.

The terminology “operations” as used in the following claims includessteps, a series of operations, actions, processes, subprocesses, acts,functions, and/or subroutines.

As used herein the terminology “succession” means the act or process offollowing in order or sequence, but is not limited to sequential order.As used herein the terminology “succession” refers to a later takenimage being compared with an earlier taken image.

As used herein the terminology “array” refers to a systematicarrangement of data in rows and columns. An example of an array is amatrix which is a rectangular array of numbers, symbols, or expressions.Examples of arrays include a matrix which is a rectangular array ofnumbers, symbols, or expressions and a vector which is a linear array ofnumbers, symbols or expressions.

As used herein, the terminology “phase” refers to a property of wavesthat measures the advance of one wave relative to another wave or thatwave at an earlier place in space-time. Quantum particles such asphotons having some wave properties may exhibit phase properties. Phasedifferences may be manifest as interference or diffraction fringes.Since images are the result of interaction of quantum particles withreflective or scattering objects they can exhibit interference fringesas signatures of instantaneous or average phase variations. Oftenobjects that exhibit some fringes can be referred to as phase objects.“Phase information” refers to images that provide indications such asinterference or diffraction fringes induced by the target and itsenvironment. Phase information can be useful to distinguish features ofthe target that are generally not as recognizable without the phaseinformation. Phase is discussed in R. E. Meyers, K. S. Deacon, and Y. H.Shih, “Turbulence-free ghost imaging,” Appl. Phys. Lett. 98, 111115(2011), R. E. Meyers, K. S. Deacon, and Y. H. Shih, “Positive-negativeturbulence-free ghost imaging,” Appl. Phys. Lett. 100, 131114 (2012). Asused herein the terminology, “SPAD” refers to an array of Single PhotonCounting Detectors that is used for imaging.

As used herein the terminology, “synchronous” means data or frames thatare acquired at the time or are time coincident.

As used herein the terminology, “asynchronous” means data or frames thatare not acquired at the same time or are not time coincident.

As used herein the terminology, “light” is meant to describeelectro-magnetic energy of any portion of the electro-magnetic spectrumto include, but not limited to, cosmic rays, x-rays, ultra-violet,visible, infra red, terahertz, microwave, and radio waves. Light may bemeasured in terms of its photons or fields.

As used herein for those embodiments that indicate two or more sensors,the sensors need not be co-located and may be distant from the othersensor(s), each sensor would have optionally present optical elementssuch as lenses or telescopes, each sensor system could optionallyconsist of a processor and memory, and further include means to exchangemeasurement values with other remotely located sensor system locations.

Super-resolution generally refers to methods and techniques that enhancethe resolution of an imaging system. This resolution increase can meanexceeding the diffraction limit ΔL=1.22*f*λ/D where ΔL is the spatialresolution, f is the focal length of the lens, X the wavelength of thelight and D the diameter of the lens aperture, or super-resolution mayinvolve extracting sub-pixel features from an image or set of imagesusing digital image processing techniques.

For the current invention, one way to enhance the resolution is todistribute the measured intensity value at pixels onto a finer scalearray of pixel. For example, a 2×2 array of pixels could be expanded toa 4×4 pixel array. Each pixel of the coarse 2×2 array may be partitionedinto another 2×2 pixel where each pixel of the fine scale array would beapportioned ¼ of the value of the parent coarse pixel. This type ofdistribution is sometimes referred to as “injection”. Another method todistribute values to a finer grid from a coarse grid would involveinterpolation techniques such as bilinear interpolation where valuesinterior to four surrounding points are a bounded linear combination ofthe values at the four surrounding points.

The invention can be used with measurements of quantum particles. Thereare many quantum particles including but not limited to photons,electrons, neutrons, protons, atoms, ions, mesons, and positrons.Photons, mesons, neutral atoms or molecules are bosons. Fermions includequantum particles such as electrons, ionized atoms and ionized moleculessometimes referred to as ions.

The invention can be used to generate improved images through a varietyof gaseous, solid, or liquid media or mixtures of these that are atleast partially transparent to quantum particles. These media mayinclude but are not limited to, for instance, glasses, diamond, silicon,water and air. As an example, images captured with an underwater cameracan be used as input for the inventive process for enhancement as wellas images taken through say an air-water interface such as an imagingsystem on a boat looking down into the water or a submerged imaginglooking into the air above the water.

Although various preferred embodiments of the present invention havebeen described herein in detail to provide for complete and cleardisclosure, it will be appreciated by those skilled in the art thatvariations may be made thereto without departing from the spirit of theinvention.

It should be emphasized that the above-described embodiments are merelypossible examples of implementations. Many variations and modificationsmay be made to the above-described embodiments. All such modificationsand variations are intended to be included herein within the scope ofthe disclosure and protected by the following claims. The invention hasbeen described in detail with particular reference to certain preferredembodiments thereof, but it will be understood that variations andmodifications can be effected within the spirit and scope of theinvention. All references mentioned herein are hereby incorporation byreference herein.

What is claimed is:
 1. A processor implemented method for imageimprovement comprising: receiving a plurality of frames of a givenregion of interest, the frames comprised of a plurality of pixels;determining, based on a quantum property of the frames, a normalizedpixel intensity value for each pixel of each of the plurality of frames;and generating an improved image of the given region of interest basedon the plurality of frames and the corresponding normalized pixelintensity values for the frames, the order of the image being two. 2.The method of claim 1, wherein determining the normalized pixelintensity value of a frame comprises: determining pixel values within aframe; summing pixel intensity values for determined pixel values withina frame; and dividing each summed pixel value by the number ofdetermined pixel values to form a normalized pixel intensity value foreach pixel in the frame.
 3. The method of claim 2, wherein generatingthe improved image of the region of interest comprises: calculating (i)the average of the product of determined pixel values and thecorresponding normalized pixel intensity values for the plurality offrames, and (ii) the product of the average of the determined pixelsvalues for each frame and the average of normalized pixel intensityvalues for the plurality of frames.
 4. The method of claim 3, furthercomprising: taking the difference of (i) and (ii).
 5. The method ofclaim 3, wherein calculating (i) the average of the product ofdetermined pixel values and the corresponding normalized pixel intensityvalues for the plurality of frames comprises: multiplying pixel valuesfor determined pixels within each frame by the corresponding normalizedpixel intensity values for that frame to produce a product for eachframe; summing the products of all the frames; and determining theaverage of first product arrays by dividing the sum of product by thenumber of frames.
 6. The method of claim 3, wherein calculating (ii) theproduct of the average of the determined pixels values for each frameand the average of normalized pixel intensity values for the pluralityof frames comprises: determining the average value of each pixel foreach frame for the plurality of frames; determining the averagenormalized pixel intensity value for each pixel for the plurality offrames; and multiplying the average pixel values and the average of thenormalized pixel intensity value for each pixel.
 7. The method of claim2, wherein determining pixels values within a frame comprises: selectingall pixels within each frame; selecting pixel based upon at least onepredetermined criterion; selecting pixels which are shifted apre-determined distance away from select pixels; and/or determining anaverage value of adjacent pixels for select pixels.
 8. The method ofclaim 1, further comprising: selecting at least one measureable propertyfor determining a normalized pixel intensity value for each pixel ofeach of the plurality of frames; and using at least one differentmeasurable property of the plurality of frames for generating theimproved image.
 9. The method of claim 8, wherein a measureable propertycomprises: wavelength or wavelength band, color, polarity, polarization,orbital angular momentum, spin, a quantum particle; or any combinationthereof.
 10. The method of claim 1, wherein the frames comprise regionsof interest that are radiation emitting.
 11. The method of claim 1,wherein the frames of a region of interest comprise sparse image data,and wherein the improved image is generated using the sparse image data.12. The method of claim 1, further comprising: determining a frameintensity deviation value for each frame by subtracting the averageframe intensity for the plurality of first frames from the frameintensity for each frame; and classifying the frame intensity deviationvalues for each frame based on whether the frame intensity deviationvalues is positive or negative.
 13. The method of claim 12, furthercomprising: selecting processing for generating an improved image basedon said classification.
 14. The method of claim 13, further comprising:calculating one or more conditional product values of the classifiedframe intensity deviation values for each frame; and selecting one ormore of the conditional product values to generate the improved image.15. The method of claim 14, wherein at least two calculated conditionalproduct values are treated differently based upon their classification.16. The method of claim 14, wherein all calculated conditional productvalues are used to generate the improved image without any changethereto.
 17. The method of claim 1, further comprising interpolating thepixel values for each frame to a finer resolution.
 18. The method ofclaim 1, further comprising filtering the frame data, the normalizedpixel intensity value, and/or any data used in one or more calculationsthereof.
 19. The method of claim 1, further comprising: providing aniterated improved image of the region of interest which comprises:specifying one or more pixel locations to be normalized to form thenormalized pixel intensity value for each pixel; selecting new pixellocations based on a pre-determined pixel selection criteria from thevalues of the improved image of the region of interest; reviewing thenew determined pixels to determine if the new determined pixel locationsare substantially the same as the pixel locations previously determinedpixel locations; and repeating the aforementioned steps until aspecified iteration criteria is met.
 20. The method of claim 1, furthercomprising using fewer than the total number of frames to determine animproved image of the region of interest.
 21. A system for imageimprovement comprising at least one processor; at least one input forreceiving or inputting frames of data; and at least one memoryoperatively associated with the at least one processor adapted to storeframes of data taken of a region of interest, each frame of datacomprising an array of pixels, each pixel having a pixel value, whereinthe at least one processor is configured to process a plurality offrames of a given region of interest according to the method of claim 1.22. A processor implemented method for image improvement comprising:receiving a plurality of frames of a given region of interest, theframes comprised of a plurality of pixels, each pixel including a valueof at least one measureable property of quantum particles; specifyingthe order of the improved image to be generated, the order being greaterthan or equal to two; selecting at least one measureable quantumproperty for pixel values of the frames corresponding to the specifiedorder; determining, based on the at least one measureable quantumproperty, normalized pixel intensity values for each pixel of each ofthe plurality of frames up to the specified order, to generate theimproved image; and generating an improved image of the given region ofinterest based on the plurality of frames and the correspondingnormalized pixel intensity values for the frames.
 23. The method ofclaim 22, wherein the at least measureable quantum property comprises:wavelength or wavelength band, color, polarity, polarization, orbitalangular momentum, spin, quantum phase, a quantum particle; or anycombination thereof.
 24. A system for generating an image of a targetilluminated by quantum entangled particles comprising: an illuminationsystem comprising: at least one source of quantum entangled particlepairs; and a beamsplitter receiving the quantum entangled particlepairs, such that one particle from each pair of particle generated byeach source interfere on the beamsplitter causing the interferedparticles to be directed towards a target and the remaining particlepairs are not directed towards the target, wherein the illuminationsystem is configured so that the interfered particles interact with thetarget; a measuring system comprising a first detector and a seconddetector that are configured to perform at least one spatially resolvedmeasurement of particles, where the first detector measures one of theremaining particle pairs and the second detector measures the other ofthe remaining particle pairs; and a processor configured to generate animage of the target based upon the correlated measured values andspatial information from the first detector and the second detector. 25.The system of 24, further comprising: electronics configured todetermine coincidences based on measurements of the first and seconddetectors which occur within a predetermined time interval.
 26. Thesystem of 25, wherein the processor is configured to generate at least asecond order image using the coincidences.
 27. The system of 25, whereinthe processor is configured to apply an image improvement method forgenerating at least a second order image using at least one measureablequantum property.
 28. The system of 24, further comprising an opticaldelay element configured to introduce a time delay for particlesreaching the measuring system.
 29. The system of claim 28, wherein theoptical delay element is further configured to be operated so as togenerate an absorption image of the target, a reflection image of thetarget, or both.
 30. The system of claim 24, wherein the illuminationsystem comprise: a single source of entangled particle pairs; and a pairof beamsplitters receiving the entangled particles pairs, such that oneparticle from each pair of particles generated by each source interfereon the beamsplitter causing the interfered particles to be directedtowards a target and the remaining particle pairs to be retained;wherein the illumination system is configured so that the interferedparticles interact with the target causing absorption at the targetentangling the retained particle pairs.
 31. The system of claim 24,wherein the system is configured so that the interfered particlesinteract with the target causing absorbtion at the target entangling theretained particle pairs.
 32. The system of claim 24 wherein the systemis configured so that the interfered particles interact with the targetcausing reflection at the target and further comprises optics orfocusing components and measurement electronics wherein the measurementof the reflected entangled particles entangled the retained particlepairs.